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I.        INTRODUCTION 

Decision  support  systems  (DSS)  have  been  available  for  a  number  of  years.  Tradi- 
tional systems  were  typically  large  mainframe  applications  developed  for  a  specific  pur- 
pose such  as  airline  reservations  or  inventory  control.  In  recent  years,  with  the  prolifera- 
tion of  desktop  computers,  a  growing  number  of  DSS  generators  have  become  available 
that  enable  the  manager  to  easily  tailor  applications  to  specific  and  non-recurring  deci- 
sions. Additionally,  the  low  cost  of  some  packages  makes  them  ideal  instructional  tools  in 
academic  courses  that  teach  decision  analysis. 

The  objective  of  this  research  is  two  fold.  This  thesis  will  attempt  to  establish  the 
features  and  characteristics  a  prospective  user  should  consider  when  selecting  a  DSS  gen- 
erator   Since  the  appropriateness  of  a  given  DSS  depends  on  its  intended  use,  we  will 
consider  three  potential  uses:  educational,  production,  and  demonstration.  In  each  of 
these  environments  different  features  of  a  DSS  tool  are  desirable.  A  DSS  used  in  a  pro- 
duction setting  would  be  more  useful  if  it  supports  linkage  with  databases  of  various  for- 
mats and  sizes,  whereas,  a  DSS  used  in  the  classroom  would  likely  use  only  small  demon- 
stration data  sets  which  can  be  constructed  as  needed.  For  demonstration  use,  power  and 
flexibility  may  be  less  important  than  the  ability  to  quickly  build  a  working  model.  Certain 
features  such  as  Windows  and  mouse  interface,  online  help,  and  logical  construction  of 
decision  problems  are  desirable  in  all  DSS  packages. 

Second,  having  determined  what  features  are  desirable  in  a  DSS,  this  thesis  will  re- 
view some  of  the  most  popular  packages  currently  available  detailing  their  specific  advan- 
tages and  shortcomings   Due  to  the  rapidly  evolving  nature  of  computer  software,  this 
thesis  will  avoid  specific  recommendations  for  use  of  software  packages,  focusing  instead 
on  what  issues  the  prospective  user  should  consider  when  selecting  a  DSS. 

Chapter  II  presents  an  overview  of  decision  support  systems,  decision  processes, 
and  methods  for  decision  making   Readers  who  are  conversant  with  DSS  can  disregard 
this  chapter.  It  is  included  as  background  material  for  those  less  familiar  with  DSS. 


Chapter  IE  discusses  the  important  features  which  a  prospective  user  should  con- 
sider when  selecting  a  DSS.  Characteristics  included  features  relating  to  the  three  compo- 
nents including  the  user  interface,  the  database  management  system  and  the  model  man- 
agement system.  Desktop  DSS  generators  are  most  capable  in  the  user  interface  area, 
making  effective  use  of  the  Windows  and  mouse  interface  of  a  PC.  Most  desktop  systems 
lacked  extensive  database  and  model  management  features. 

Chapter  IV  discusses  considerations  as  they  relate  to  specific  uses.  Different  DSS 
features  are  more  important  to  some  users  than  others.  In  a  production  setting,  DSS  are 
perhaps  more  practical  for  non-recurring  strategic  decisions.  The  limited  exchange  capa- 
bilities of  today's  products  make  them  less  suited  for  recurring  decisions  which  may  rely 
on  external  data.  DSS  can  make  excellent  supplements  to  classes  which  teach  DSS.  They 
can  be  used  either  to  teach  DSS  or  to  demonstrate  interactively  the  concepts  of  decision 
trees  or  multi-criteria  decision  making. 

Chapter  V  highlights  some  of  the  more  popular  packages  based  on  the  considera- 
tions developed  in  Chapters  HI  and  IV.  Chapter  VI  offers  conclusions  and  recommenda- 
tions for  further  research. 


II.       BACKGROUND:  DECISION  MAKING  AND  DSS 

A.         DECISION  SUPPORT 

Decision  support  is  more  than  option  selection.  In  practice,  decision  support  con- 
sists of  gathering  information,  prioritizing,  filtering,  the  identification  of  decision  options, 
the  evaluation  and  implementation  of  options,  and  the  evaluation  of  the  decision  process 
itself,  which  can  occur  before,  during  or  after  the  decision  making  cycle.  [Andriole,  1989] 

Decision  support  systems  are  defined  as  interactive  computer  programs  that  utilize 
analytical  methods,  such  as  decision  analysis,  optimization  algorithms,  program  scheduling 
routines,  and  so  on,  for  developing  models  to  help  decision  makers  formulate  alternatives, 
analyze  their  impacts,  and  interpret  and  select  appropriate  options  for  implementation. 
[Adelman,  1992] 

In  the  1 970' s,  decision  support  systems  were  written  to  support  specific  recurring 
problems  and  did  not  support  reuse.  These  programs  were  typically  custom  designed  for 
one  application.  In  thel980's,  DSS  generators  were  introduced  which  allowed  a  user  to 
custom  tailor  an  application  to  a  given  decision.  These  programs  typically  lacked  in  vari- 
ous capabilities  such  as  their  user  interface  or  data  handling  capabilities. 

The  decreasing  cost  of  hardware  and  the  increasing  power  of  software  in 
the  1 990 's  make  it  possible  for  decision  makers  to  use  sophisticated  problem-solving  tech- 
niques. Off-the-shelf  DSS  generators  are  much  more  affordable  than  they  were  just  a  few 
years  ago.  Many  generators  are  now  available  for  personal  computers,  thereby  decreasing 
the  implementation  costs  and  problems.  They  make  use  of  the  user  friendly  Windows  in- 
terface and  offer  increased  data  handling  capabilities  which  make  them  more  appealing  to 
the  typical  manager  with  a  limited  management  science  background. 

1.         Benefits 

There  are  clear  commercial  successes  from  the  use  of  DDS.  For  example,  a  Cana- 
dian utility  company  uses  a  desktop  DSS  to  determine  when  to  deploy  repair  crews  during 


minimally  staffed  periods.  A  southwestern  power  company  used  a  desktop  DSS  to  select 
among  various  options  for  increasing  power  generation  capability. 

Improved  organizational  effectiveness  can  occur  in  many  ways,  such  as  through 
decreased  personnel  costs,  greater  access  to  expert  knowledge,  or  improved  decision 
making  [Adelman,  1992].  Perhaps  the  greatest  benefit  of  using  a  desktop  DSS  is  not 
having  a  software  package  to  solve  the  problem  at  hand  but  forcing  the  decision  maker  to 
define  alternatives  and  work  through  a  decision  logically.  Decision  analysis  is  the  disci- 
pline of  evaluating  complex  alternatives  in  terms  of  values  (what  we  care  about)  and  un- 
certainty (what  we  know  and  do  not  know).  Experienced  decision  analysts  and  educators 
stress  that  the  benefits  of  decision  analysis  are  insight  into  how  the  defined  alternatives 
differ  from  one  another,  and  generating  suggestions  for  new  and  improved  alternatives. 
Too  many  critics  stress  the  use  of  numbers  to  quantify  subjective  values  and  uncertainties 
without  realizing  the  power  of  quantitative  analysis  for  generating  qualitative  insight 
[Booed,  1996]. 

2.         Limitations 

In  spite  of  the  successes,  the  "state  of  the  art"  has  not  matched  the  "state  of  the 
expectation."  The  fact  is  that  many  decision  support  systems  are  simply  not  used.  Ven- 
dors have  vested  interests  in  overselling,  and  users  are  inclined  to  want  to  believe  that  a 
solution  to  all  their  problems  can  be  found  on  one  or  two  floppy  disks.  Andriole  [p.  7] 
points  out,  that  the  state  of  the  art  of  decision  support  systems  technology  is  unbalanced 
and  evolving  [Andriole,  1989]. 

Managers  do  not  use  models  as  much  as  they  could.  Models  require  accurate  data 
that  may  be  time  consuming  and  costly  to  produce.  Even  if  a  good  model  is  developed 
and  accurate  data  obtained,  a  manager  may  not  understand  it,  preferring  their  own  simple 
analysis  to  a  complicated  model.  Lastly,  even  when  managers  do  use  a  model  and  imple- 
ment the  results,  they  may  not  realize  that  this  is  what  they  have  done. 

In  order  to  utilize  such  supporting  software  effectively,  the  user  needs  some  under- 
standing of  the  fundamental  concepts  of  models  and  of  decision  support  concepts.  The 


existence  of  user  friendly  modeling  makes  it  easy  to  create  bad  models  or  to  misuse  good 
models  [Young,  1989]. 


B. 


DECISION  MAKING 


Decision  making  is  a  basic  skill  required  in  any  managerial  position.  It  is  only  in 
the  last  25  years,  however,  that  management  theorists  have  focused  on  decision  making  as 
a  science.  This  section  gives  an  overview  of  decision  making,  the  importance  of  the  envi- 
ronment in  which  the  decision  is  being  made,  several  strategies  for  making  decisions,  and 
the  decision  making  process. 


1. 


Decision  Environment 


There  are  a  number  of  ways  to  view  the  environment  in  which  a  decision  is  made. 
Of  primary  importance  in  establishing  environmental  considerations  is  at  what  level  in  the 
organization  the  decision  is  being  made.  Figure  1  shows  the  three  principle  levels  of  deci- 
sion making,  strategic,  tactical  and  operational. 


Unstructured 


Structure 


Long  Term  Impact 


Short  Term  Impact 


Figure  1  Levels  of  decision  making 

At  the  strategic  decision  making  level,  top  executives  develop  overall  organiza- 
tional goals,  strategies,  policies,  and  objectives  through  long-range  strategic  planning. 
They  also  monitor  the  strategic  performance  of  the  organization  and  its  overall  direction. 
At  the  tactical  level,  middle  level  managers  develop  short  and  medium  range  plans  and 
budgets,  and  specify  the  policies,  procedures,  and  objectives  for  units  of  the  organization. 


They  also  acquire  and  allocate  resources  and  monitor  the  performance  of  organizational 
units.  At  the  operational  level,  supervisory  managers  develop  short  range  planning  de- 
vices such  as  production  schedules.  They  direct  the  use  of  resources  and  the  performance 
of  tasks  according  to  established  procedures  and  within  budgets  established  for  them. 

The  level  at  which  the  decision  is  being  made  typically  dictates  the  environmental 
factors  which  influence  decision  making.  These  factors  include  the  type  of  stimuli,  the  de- 
gree of  coordination,  the  nature  of  the  task,  and  the  degree  of  uncertainty. 

The  type  of  stimuli  refers  to  why  the  decision  is  being  made.  At  a  higher  level,  the 
stimulus  is  more  controlled  and  may  involve  taking  advantage  of  an  unforeseen  opportu- 
nity. At  the  lower  level,  decisions  are  more  focused  on  solving  urgent  problems  such  as 
allocating  personnel  and  resources. 

The  requirement  for  coordination  increases  at  higher  levels  in  the  organization.  At 
the  lower  levels  of  an  organization,  decisions  tend  to  be  more  individual  such  as  a  unit  su- 
pervisor planning  the  allocation  of  resources  for  a  short  time  frame.  At  a  higher  level,  ex- 
ecutives may  make  group  decisions  using  data  pulled  from  many  sources  both  inside  and 
outside  the  organization. 

Uncertainty  in  decision  making  also  increases  as  the  organizational  level  increases. 
This  is  only  logical.  At  the  lower  levels  of  an  organization,  decision  makers  face  some- 
what routine  decisions  using  limited  amounts  of  data.  At  strategic  levels  executives  are 
faced  with  decisions  which  they  have  never  made  before  and  with  data  pulled  from  multi- 
ple internal  and  external  sources. 

Nature  of  task  refers  to  whether  the  task  is  unstructured,  semi-structured,  or 
structured.  Structured  decisions  involve  situations  where  the  procedures  to  follow  when  a 
decision  is  needed  can  be  specified  in  advance.  Therefore,  such  decisions  are  structured  or 
programmed  by  the  decision  rules  developed  for  them.  A  structured  decision  may  involve 
what  is  known  as  a  deterministic  or  algorithmic  decision.  In  this  case,  a  decision's  out- 
come can  be  determined  with  certainty  if  a  specified  sequence  of  activities  (an  algorithm) 
is  performed.  An  example  of  such  a  system  is  the  Navy's  Uniform  Inventory  Control 


Point  (UICP)  system  which  runs  at  the  Navy  Inventory  Control  Point.  This  system  re- 
ceives a  requisition  for  material  from  a  Navy  unit,  and  then  passes  the  requisition  to  a  sup- 
ply point  for  material  issue  based  on  such  factors  as  the  on-hand  balance  of  the  item  at  the 
various  supply  points.  The  rules  regarding  filling  outstanding  requisitions  are  programmed 
into  UICP's  B01  module  and  will  result  in  a  predictable  decision  being  made  for  a  set 
group  of  inputs. 

Unstructured  decisions  involve  decision  situations  where  it  is  not  possible  or  desir- 
able to  specify  in  advance  most  of  the  decision  procedures  that  will  follow.  Many  decision 
situations  in  the  real  world  are  unstructured  because  they  are  subject  to  too  many  random 
or  changeable  events  or  involve  too  many  unknown  factors.  At  most,  many  decision 
situations  are  semi-structured.  Decisions  involved  in  starting  a  new  repair  line  at  a  Naval 
Aviation  Depot,  for  example,  would  range  from  unstructured  to  semi-structured.  The 
many  unknown  factors  involved  would  require  a  less  structured  approach  leading  to  sub- 
jective judgements  by  managers.  Information  systems  can  support  such  decisions  by  pro- 
viding the  ability  to  make  ad  hoc  inquiries  for  information  in  databases  and  the  ability  to 
reach  a  decision  in  an  interactive  process  using  a  DSS. 

Decisions  are  also  broken  down  between  nonrecurring  decisions,  also  called  ad 
hoc,  and  recurring,  also  called  institutional.  The  type  of  decision  being  made  dictates 
many  of  the  features  which  are  required  of  the  DSS  as  shown  in  Table  1 . 


Ad  hoc  vs.  Institutional  use: 

Institutional 

Ad  Hoc 

Decision  occurrences  for  a  decision  type 

Many 

Few 

Decision  types 

Few 

Many 

Number  of  people  making  decisions  of  same  type 

Many 

Few 

Range  of  decisions  supported 

Narrow 

Wide 

Range  of  users  supported 

Narrow 

Wide 

Specific  data  known  in  advance 

Usually 

Rarely 

Problems  recurring 

Usually 

Rarely 

Importance  of  operational  efficiency 

High 

Low 

Duration  of  specific  type  of  problem  being  addressed 

Long 

Short 

Need  for  rapid  development 

Low 

High 

Table  1  [From  Sprauge,  1996] 

2.  Decision  Strategies 

There  are  several  avenues  of  approach  that  a  decision  maker  can  utilize  in  order  to 
reach  a  decision.  Different  people  will  use  different  strategies  at  different  times  for  differ- 
ent kinds  of  decisions.  Which  strategy  to  select  depends  on  factors  such  as  what  is  the 
information  processing  requirements  for  each  decision  making  strategy  and  which  strategy 
the  decision  maker  prefers.  Several  strategies  will  be  discussed  including  optimizing,  sat- 
isficing,  and  selection  by  elimination. 


a.         Optimizing 

The  goal  when  optimizing  is  to  select  the  course  of  action  with  the  highest 
payoff.  Using  this  technique  requires  the  decision  maker  to  estimate  all  costs  and  benefits 
of  every  viable  course  of  action.  In  even  a  simple  decision  making  scenario,  optimizing 
can  be  an  overwhelming  task.  In  many  cases,  there  is  too  much  information  to  process 
and  too  many  variables  to  consider  simultaneously.    This  can  result  in  a  high  cost  in  time, 
effort  and  money.  Additionally,  optimization  of  stated  objectives  may  result  in  sub- 
optimization  on  unstated,  less  tangible  objectives.  As  a  result  of  the  difficulty  with  the 
optimization  technique,  decision  makers  frequently  do  not  consider  all  possible  alterna- 
tives, do  not  consider  all  objectives  and  criteria  or  place  more  weight  on  intangible  objec- 
tives. 


Many  desktop  DSS  generators  perform  "comparative  optimization".  The 
user  declares  the  alternatives,  and  the  programs  helps  the  user  compare  them.  Standard 
optimization  tools  (for  linear  programming)  generate  the  alternatives  from  an  abstract  de- 
scription of  the  constraints  and  test  them. 

b.         Satisficing 

Due  to  the  complexities  inherent  in  optimizing,  decision  makers  often  sat- 
isfice    They  choose  courses  of  action  that  are  "good  enough"  that  meet  a  certain  minimal 
set  of  requirements.  Multiple  criteria  are  often  used,  but  the  "satisfactoriness"  of  an  out- 
come is  the  decisive  factor.  Often,  uncertainty  about  the  best  choice  makes  decision  mak- 
ers gravitate  towards  a  more  conventional  second  best  choice. 

a  Quasi-Satisficing 

Quasi-satisficing  is  usually  used  in  cases  of  crisis  management  or  when  the 
decision  involves  someone  else's  problem.  It  is  applicable  when  there  are  fewer  objectives 
that  need  to  be  met  or  when  very  few  alternatives  are  generated.  Quasi-satisficing  can  be 
summarized  as  "do  something  useful  without  deliberating  about  all  other  alternatives,"  or 
"do  what  we  did  the  last  time  if  it  worked,  or  the  opposite  of  if  it  did  not." 

d.  Selection  by  Elimination 

Selection  by  elimination  involves  eliminating  alternatives  that  do  not  meet  , 
the  most  important  criterion.  Decision  making  using  this  strategy  becomes  a  sequential 
narrowing  down  process.  The  disadvantage  of  this  technique  is  that  if  improper  weights 
are  assigned  to  criteria,  a  "better"  alternative  might  be  eliminated  early  on.  In  addition,  for 
more  complex  problems,  this  process  might  still  leave  the  decision  maker  with  a  large 
number  of  alternatives. 


3.  Decision  Process 

The  decision  making  process  is  generally  considered  to  consist  of  three  phases: 
intelligence,  design,  and  choice.  Intelligence  involves  the  identification  of  a  problem  that 
requires  a  decision.  Design  involves  the  creation  and  evaluation  of  alternative  courses  of 
action.  Choice  is  the  selection  of  a  course  of  action. 

DSS  typically  addresses  a  specific  problem  that  has  been  identified.  Thus,  it  is  of- 
ten unnecessary  to  support  the  intelligence  phase,  especially  for  decisions  at  a  lower  lev- 
els. 

The  ability  to  support  the  design  phase  of  decision  making  is  the  true  test  of  DSS. 
The  core  of  any  DSS  is  the  model  base  which  has  been  built  to  analyze  a  problem  or  deci- 
sion. The  primary  value  to  a  decision  maker  of  a  DSS  is  the  ability  of  the  decision  maker 
and  the  DSS  to  explore  the  models  interactively  as  a  means  to  identify  and  evaluate  alter- 
native courses  of  actions.  This  is  of  tremendous  value  to  the  decision  maker  and  repre- 
sents the  DSS  capability  to  support  the  design  phase.  [Sprague,  1996] 

The  choice  phase  of  decision  making  is  the  most  variable  in  terms  of  support  from 
DSS.  Although  a  custom  designed  DSS  can  be  programmed  to  make  decisions,  DSS  gen- 
erators do  not  make  decisions  y  them  selves.  They  do  however,  present  the  user  with  a 
detailed  analysis  of  the  factors  influencing  the  decision.  In  this  regard  they  support  the 
choice  phase. 

C.         RELATED  TECHNOLOGIES 

There  are  a  number  of  other  types  of  programs  that  are  similar  to  decision  support 
systems  and  which  are  commonly  grouped  with  DSS  when  teaching  decision  support  or 
thinking  about  decision  making.  These  programs  include  executive  information  systems, 
expert  systems,  and  group  DSS. 

1.  Executive  Information  Systems 

Executive  Information  systems  (EIS)  typically  are  used  to  obtain  and  provide  top 
managers  with  status  information  from  various  internal  and  external  sources.  Information 
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is  presented  in  a  wide  variety  of  forms  (textual,  graphical,  tabular)  often  on  the  same 
screen.  An  example  of  such  a  system  was  developed  by  Sherwin  Williams  Company. 
Their  system,  developed  over  the  course  of  two  years,  was  designed  to  replace  a  paper  re- 
porting system  which  ran  from  individual  stores,  through  the  regional  management  to  the 
corporate  headquarters.  The  new  system  uses  a  graphical  interface  and  allows  managers 
at  all  levels  to  review  sales  data  of  units  under  their  control.  [Booker,  1993] 

2.  Group  DSS 

One  of  the  most  popular  forms  of  decision  support  systems  being  developed  today 
is  the  Group  DSS  (GDSS).  Group  DSS  focus  on  group  activities  such  as  discussions  and 
meeting.  The  systems  are  designed  to  provide  computer  support  for  groups  of  people 
who  work  on  collaborative  projects.  GDSS  can  play  an  important  role  in  today's  chang- 
ing management  environment.  They  can  keep  people  in  better  touch  with  each  other,  sup- 
port broader  spans  of  control,  and  directly  connect  members  of  different  organizations. 
GDSS  includes  a  wide  variety  of  products  and  are  also  called  groupware,  collaborative 
work  support  systems.  [Sprague,  1996] 

3.  Expert  Systems 

Expert  systems  employ  heuristics  and  qualitative  reasoning  to  simulate  the  behav- 
ior of  an  expert,  based  on  knowledge  accumulated  by  experts.  An  example  of  an  Expert 
System  would  be  a  program  to  help  a  medical  professional  diagnose  an  injured  patient  or 
to  help  a  technician  trouble  shoot  a  piece  of  equipment. 

4.  Software 

There  are  a  number  of  software  packages  available  which  executive  information 
systems,  group  DSS,  expert  systems,  as  well  as  forecasting.  EIS  are  available  from  a 
number  of  companies    Comshare  from  Commander  EIS  or  Pilot  from  Command  Center 
allow  users  to  develop  custom  EIS  designed  to  take  advantage  of  unique  concerns  of 
management    Group  DSS  are  available  from  numerous  vendors  including  Team  Expert 
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Choice  by  Expert  Choice  and  GroupSystems  developed  by  the  University  of  Arizona. 
Like  EIS,  Expert  systems  are  frequently  custom  designed  for  a  specific  organization.  De- 
velopment tools  include  programs  such  as  VP  Expert,  SuperExpert,  and  KnowledgePro  as 
well  as  programming  languages  such  as  C++.  Forecasting  programs  are  available  includ- 
ing such  titles  as:  Minitab  Statistical  Software  by  Minitab  Inc.,  SAS  Software  from  SAS 
Institute  Inc.,  Statgraphics  Plus  for  Windows  by  Statistical  Graphics  Corp.,  and  SPSS  7.5 
for  Windows  by  SPSS  Inc. 

Reviews  of  these  types  of  program  can  be  found  in  various  information  technology 
related  journals  including  OR/MS,  Datamation,  and  others. 

D.         SUMMARY 

This  chapter  has  given  a  brief  overview  of  decision  support  systems,  their  benefits 
and  their  limitations.  Traditional  DSS  differs  from  DSS  generators  in  that  the  later  can  be 
easily  tailored  to  specific  situations.  Appendix  A  provides  additional  information  on  vari- 
ous methods  of  decision  analysis  including  decision  making  under  uncertainty  and  decision 
making  with  multiple  criteria.  Readers  unfamiliar  with  these  techniques  should  review  this 
appendix  prior  to  proceeding  to  the  next  chapter. 
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III.     DESCISION  SUPPORT  SYSTEM  FEATURES 

A.  INTRODUCTION 

When  evaluating  decision  support  systems,  it  is  important  to  remember  the  obvi- 
ous, which  is  that  the  overall  aim  of  these  systems  is  to  improve  the  effectiveness  of  the 
organization  using  them.  The  first  requirement  to  fulfilling  this  aim  is  to  determine  the 
correct  type  of  package  for  the  users  need.  After  a  review  of  the  methodology  used  for 
this  thesis,  this  chapter  discusses  some  of  the  important  characteristics  on  DSS  generators 
may  differ.  Issues  include  such  things  as  the  technical  considerations  and  the  components 
of  a  DSS.  Chapter  IV  discusses  issues  tailored  to  different  categories  of  users. 

B.  METHODOLOGY 

To  determine  which  features  are  important  in  DSS  selection  the  research  for  this 
thesis  involved  a  review  of  current  literature  on  DSS    Material  was  drawn  from  texts, 
magazines,  journals  and  the  World  Wide  Web.  Most  texts  and  papers  written  about  DSS 
have  similar  recommendations  regarding  writing  or  selecting  DSS  packages. 

In  order  to  classify  features  important  to  the  three  categories  of  DSS  users,  an  in- 
formal brainstorming  session  was  undertaken  to  group,  classify,  and  further  develop  the 
items  on  the  list  developed  from  literature  review.  From  this  list,  we  developed  a  survey 
which  was  sent  to  over  100  educators  and  professionals,  as  well  as  to  DSS  software  com- 
panies  A  separate  survey  was  developed  for  production  users,  educational  users,  and 
vendors.  Names  were  drawn  from  the  online  database  of  the  Institute  for  Operations  Re- 
search and  Management  Science  (INFORMS)  and  from  a  search  of  the  World  Wide  Web. 
Responses  were  received  from  35  percent  of  those  surveyed.  The  aim  of  the  survey  was 
to  validate  and  refine  the  list  obtained  through  brainstorming  and  to  gain  insight  into  other 
features  DSS  users  felt  were  important.    The  surveys  asked  users  to  assign  weights  to  the 
features  of  DSS,  assigning  a  higher  score  to  features  they  felt  were  most  important.  Addi- 
tional dialog  was  held  with  DSS  vendors  and  some  users  to  explore  questions  raised  by 
the  survey.  Detailed,  anonymous  responses  to  the  survey  are  shown  in  Appendices  F,  G, 
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and  H.  Appendix  I  provides  comparison  between  the  average  scores  on  the  detailed  sur- 
veys. 

The  survey  consisted  of  seven  questions.  The  first  two  questions  were  designed  to 
gather  background  data.  Educators  were  asked  about  the  level  at  which  they  taught  DSS 
and  the  role  of  DSS  software  in  their  class.  Designers  were  asked  about  the  number  of 
copies  of  DSS  software  the  company  had  produced.  All  users,  production  and  educa- 
tional, were  asked  about  packages  used,  or  rejected  for  use,  and  the  reasons  for  doing  so. 
Designers  were  also  asked  who  they  saw  as  their  primary  customers  and  how  they  best 
saw  DSS  software  being  used  for  strategic  or  operational  use. 

Question  three  regarded  criteria  to  be  considered  when  selecting  DSS  software. 
Survey  recipients  were  asked  to  allocate  100  points  between  five  criteria  including  the  im- 
portance of  the  length  of  time  required  to  learn  to  use  the  software,  the  ease  of  use,  the 
availability  of  versions  of  the  software  to  operate  on  multiple  platforms,  the  ability  to  use 
the  software  in  a  network  environment,  and  the  cost  of  acquiring  the  software. 

Question  four  asked  questions  related  to  application  features.  Again,  survey  re- 
cipients were  asked  to  allocate  100  points  among  ten  criteria  including  divided  among 
features  of  the  user  interface,  the  data  handling  system,  and  the  model  system. 

Question  five  asked  recipients  to  rate  the  importance  of  criteria  related  to  training 
users  to  use  the  program.  Recipients  were  asked  to  allocate  100  points  between  the  im- 
portance of  sample  exercises,  online  tutorials,  explanations  of  results,  and  tips  and  in- 
sights. 

Question  six  asked  recipients  to  rank  the  importance  of  specific  DSS  considera- 
tions included  in  a  software  package,  including  the  ability  to  modeling  uncertainty,  to  han- 
dle subjective  data,  to  work  with  utility  functions,  and  to  measure  risk  and  sensitivity. 

Lastly,  question  seven  asked  the  users'  questions  related  to  support  issues  includ- 
ing the  availability  of  FAQ  files,  discussion  groups,  mail  lists,  technical  support  via  phone 
or  email,  local  support,  and  documentation. 
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C.         GENERAL  DSS  CONSIDERATIONS 

This  section  discusses  the  general  considerations  for  selecting  a  DSS.  Criteria  in- 
clude what  type  of  decision  the  user  is  trying  to  solve,  what  aspects  of  modeling  a  par- 
ticular product  addresses,  and  the  technical  requirements  of  the  system. 

1.  Problem  Type 

What  DSS  problems  does  the  product  address?  There  is  no  one  DSS  package  that 
is  well  suited  to  all  decision  problems.  Broadly,  the  DSS  packages  reviewed  for  this  thesis 
can  be  grouped  into  two  areas,  decision  making  under  uncertainty  and  decision  making  for 
multi-criteria  problems.  Decisions  involving  uncertainty  are  typically  solved  using  influ- 
ence diagrams  and  decision  trees.  Decisions  with  multiple  criteria  are  solved  using  the 
analytical  hierarchy  process.    It  is  the  responsibility  of  the  potential  user  of  these  packages 
to  determine  which  approach  is  most  appropriate.  A  third  category  of  programs  that  we 
reviewed,  the  spreadsheet  add-in,  does  not  fit  into  these  two  traditional  areas.  Though 
not  technically  a  DSS,  these  packages  are  very  useful  tools  for  the  decision  maker. 

2.  What  Aspect  of  DSS  Modeling  Does  the  Product  Address? 

In  spite  of  the  differences  between  applications,  it  is  possible  to  identify  certain 
common  aspects  of  DSS  modeling  projects.  Most  attempts  to  use  DSS  software  involve, 
for  example,  defining  model  components  such  as  decision  variables,  objectives  and  con- 
straints, gathering  data  required  by  the  model;  solving  model  instances;  reporting  and 
analyzing  results,  managing  multiple  model  versions  and  data  scenarios. 

Independent  of  the  specific  software  package  being  used,  in  order  to  create  a  deci- 
sion analysis  model,  it  is  necessary  to  create  the  model  structure  and  obtain  the  data  to 
populate  the  model  for  computation.  In  some  instances,  trained  analysts  provide  the 
model  structuring  support,  however,  the  ease  of  use  of  these  packages  allows  anyone  to 
easily  build  relationships  models.  Some  DSS  start  with  an  easy  to  use  brainstorming  fea- 
ture and  then  progress  to  a  decision  support  model. 
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Once  the  structure  and  numbers  are  in  place,  the  analysis  can  begin.  Much  more  is 
involved  than  computing  the  expected,  weighted  utility  of  each  alternative.  If  the  process 
stopped  there,  decision  makers  would  not  gain  much  insight  into  the  problem  being 
solved.  Decision  makers  must  examine  the  sensitivity  of  the  key  criteria  and  weights  and 
risk  preference  parameters.  As  part  of  the  sensitivity  analysis,  it  is  possible  to  calculate  the 
value  of  perfect  information  for  uncertainties  that  have  been  explicitly  modeled.  [Booed, 
1996].  The  more  sophisticated  DSS  should  allow  the  user  to  perform  various  forms  of 
sensitivity  analysis  and  to  investigate  decisions  in  depth. 

The  packages  reviewed  for  this  thesis  are  the  stand-alone  modeling  systems,  which 
are  intended  to  provide  the  entire  interface  to  the  formulation,  solution,  and  analysis  stages 
of  modeling.  Some  of  the  models  allow  import  and  export  of  data  but  they  lend  them- 
selves only  with  difficulty  to  automated  processes  such  as  one  that  may  be  written  with 
Delphi  or  Visual  Basic. 

3.  Technical  Considerations 

The  technical  considerations  involved  in  selecting  a  DSS  are  fairly  straight  forward 
and  will  only  be  discussed  briefly. 

•Operating  systems,  memory  requirements,  and  printer  support  —  In  most  in- 
stances, the  choice  of  operating  systems  and  the  memory  required  for  a  desktop  DSS  is 
dictated  by  a  system  that  is  already  in  place.  Chapter  IV  discusses  specific  requirements  of 
desktop  systems.  Appendix  B  lists  requirements  for  currently  available  packages. 

•Security  ~  Users  should  consider  the  sensitivity  of  the  data  that  they  are  working 
with,  especially  if  the  program  is  being  used  in  a  network  environment.  Additionally,  in- 
tegrity may  become  an  issue  if  more  than  one  person  is  working  on  the  model  or  the  data- 
base. 

•Group  versus  individual  ~  Many  DSS  which  are  being  introduced  today  are  de- 
signed to  support  group  interaction.  These  packages  contain  voting  tools  and  features 
which  allow  multiple  people  to  combine  their  input  on  decision  making.  This  thesis  con- 
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centrates  on  packages  designed  for  individual  use.  Where  group  features  are  present,  they 
are  noted. 

•Process  automation  support  —  More  sophisticated  users  may  require  the  ability  to 
link  the  decision  making  abilities  of  a  DSS  to  the  analytical  capabilities  of  a  spreadsheet  or 
to  a  database.  A  few  users  surveyed  discussed  the  desire  to  be  able  to  use  Object  Linking 
and  Embedding  (OLE)  to  control  a  decision  process. 

D.         DSS  COMPONENTS 

DSS  technology  consists  of  three  sets  of  capabilities,  the  User  Interface  Manage- 
ment System  (UTMS),  the  Data  Base  Management  System  (DBMS)  and  the  Model  Man- 
agement System  (MMS).  Figure  2  is  a  depiction  of  the  relationship.  The  UTMS,  also 
called  the  Dialog  Generation  Management  System  (DGMS)  manages  the  interface  be- 
tween the  user  and  the  rest  of  the  system.  The  DBMS  and  the  MMS  contain  the  neces- 
sary functions  to  manage  the  data  base  and  model  base,  respectively.  The  characteristics 
of  each  section  are  discussed  in  greater  detail  later  in  this  chapter. 


Data  Base 


Model  Base 


DBMS    MBMS 


DGMS 


User 


Figure  2  The  components  of  DSS. 

A  good  DSS  should  have  balance  among  the  three  capabilities.  It  should  be  easy 
to  use  to  allow  non-technical  decision  makers  to  interact  with  the  system,  it  should  have 
access  to  a  wide  variety  of  data,  and  it  should  provide  analysis  and  modeling  in  a  variety  of 
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ways.  [Sprague,  1996]  Unlike  the  early  DSS  generators  which  lacked  in  one  or  more  of 
these  areas,  the  current  desktop  systems  offer  a  reasonable  balance  between  these  three 
components. 

1.         User  Interface  Management  System 

To  a  large  extent,  to  the  user,  the  User  Interface  Management  System  is  the  DSS. 
The  sense  of  the  general  character  of  the  software,  but  not  the  detailed  operational  struc- 
ture, is  obtained  through  interaction  with  UIMS.  The  power  and  scope  of  DSS  function- 
ality can  only  benefit  a  user  if  it  is  accessible  and  comprehensible.  Ease  of  use  and  under- 
standing are  determined  largely  by  the  character  of  the  interface.  Similarly,  the  more  a 
DSS  can  do,  the  more  difficult  it  is  to  provide  an  interface  that  is  easy  to  use  and  under- 
stand. [Young,  1989]  The  UIMS  is  characterized  by  the  control  mechanisms  of  the  pro- 
gram, the  quality  of  the  display,  and  the  dialog  which  is  held  between  the  system  and  the 
user. 

a.         Control  Mechanisms 

(1)  Mouse  and  Windows  Support  ~  Respondents  to  the  thesis  sur- 
vey placed  strong  emphasis  on  the  necessity  of  having  mouse  and  Windows  support  for 
DSS  packages.  The  graphical  abilities  of  Windows  make  available  a  number  of  useful 
features  such  as  dynamic  sensitivity  analysis  which  will  be  discussed  later.  Though  most 
programs  written  today  utilize  the  Windows  interface,  some  provide  more  flexibility  than 
others.  For  example,  only  a  few  programs  make  use  of  the  right  mouse  button  which 
opens  a  pop  up  menu  showing  the  most  appropriate  features  depending  on  where  the 
mouse  is  pointing.  This  can  be  a  time  saving  feature  for  the  experienced  user. 

(2)  Visualization  and  Ingredients  of  a  Good  Display  ~  Proper  dis- 
play design  is  critical.  A  number  of  studies  have  shown  that  tasks  involving  visual  search, 
counting,  noting  of  display  changes  and  information  extraction  are  adversely  affected  by  a 
high  numbers  of  items  displayed  on  a  screen.  The  more  information  displayed  on  a  screen, 
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the  greater  the  chances  for  error,  misperception,  frustration  and  eventual  abandonment  of 
the  system.  Ingredients  to  look  for  when  evaluating  a  good  display  include: 

•  Orderly,  clean,  clutter  free  displays. 

•  Obvious  indication  of  what  is  being  shown. 

•  Expected  information  where  it  should  be. 

•  Clear  indication  of  what  relates  to  what. 

•  Plain  simple  English. 

•  Clear  indication  when  an  action  could  make  a  permanent  change  in  the  data  or 
model  design  [Andriole,  1989]. 

Many  users  find  color  input  and  output  routines  initially  pleasing, 
however,  research  suggests  that  unless  color  is  used  carefully,  its  initial  appeal  will  shortly 
be  replaced  with  frustration.  Many  color  displays  violate  some  basic  rules  of  presentation. 
First,  too  many  colors  often  appear.  Some  use  color  inconsistently.  Some  use  offensive 
combinations.  One  often  wonders  if  the  authors  of  software  use  color  to  improve  per- 
formance or  use  it  simply  because  color  display  devices  have  dropped  in  price.  Figure  4 
shows  a  display  which  violates  most  of  the  rules  for  appropriate  guidelines  for  the  use  of 
color    Some  considerations  users  should  look  for  when  evaluating  the  quality  of  a  color 
display  include: 

•  Use  color  as  a  formatting  aid  to  relate  or  tie  fields  into  groupings,  emphasize 
important  fields  and  relate  fields  that  are  spatially  separated. 

•  Use  color  as  a  visual  code  to  identify  kinds  of  data,  sources  of  data,  status  of 
data. 

•  For  emphasis  and  separations  use  contrasting  colors  such  as  red  and  green. 

•  To  convey  similarity  use  similar  colors  such  as  orange  and  yellow. 

•  Use  colors  consistently  within  and  across  systems. 
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Figure  3  This  output  uses  color  for  the  sake  of  color.  There  is  no  meaning 
behind  the  colors  selected  in  the  output.  [From  Decide  Right  by  Avantos] 

The  improper  use  of  color  will  undermine  the  human-computer 
communication.    [Andriole,  1989] 

(3)  Dialog  management  ~  Over  the  years  a  number  of  different 
dialogues  have  been  used  for  DSS.  Dialog  types  include  menues,  query  language,  func- 
tion keys  and  form  filling.  These  dialog  types  vary  in  the  complexity  they  involve  as  well 
as  the  flexibility  they  provide. 

Menus  are  frequently  used  for  command  construction  as  well  as 
data-base  search.  Pull  down  or  pop  up  menus  are  frequently  used  to  drive  Windows  based 
programs  such  as  the  ones  review  for  this  thesis.  Query  Language  is  frequently  found  in 
older  programs.  In  this  type  of  program,  the  user  enters  questions  or  data  base  access  re- 
quests and  the  system  produces  a  report.  This  is  acceptable  for  a  well-trained  user,  oth- 
erwise it  can  be  error  prone  and  frustrating.  Query  language  is  also  used  to  access  the 
help  features  of  many  Windows  based  programs  allowing  the  user  to  enter  the  topic  on 
which  help  is  desired.  Function  Keys  provide  a  short  cut  to  frequently  used  commands  and 
are  found  in  some  of  the  desktop  DSS  tools.  Function  keys  provide  flexibility  for  more 
experienced  users  of  a  program.  Form  filling  presents  a  form  with  fill-in  blanks  which  the 
user  files  in.  This  type  of  dialog  is  frequently  used  to  enter  data  or  weightings. 
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Dialog  involves  complexity  versus  flexibility  trade-offs.  For  exam- 
ple, while  the  question-answer  approach  to  data  entry  is  simple  and  often  appropriate  for  a 
novice  user  performing  well  structured  tasks,  it  does  not  provide  flexibility  beyond  what 
was  planned  by  the  systems  designers.  Menu  oriented  systems  impose  the  same  kind  of 
structure.  In  contrast,  command  languages  place  the  user  more  in  control  but  require  ad- 
ditional knowledge  to  use  the  systems.  [Sprague,  1996] 

Flexibility  is  a  measure  of  the  number  of  ways  in  which  a  user  can 
accomplish  a  given  function  and  can  be  achieved  by  providing  a  large  number  of  com- 
mands.   There  is  evidence  that  less  experienced  users  tend  to  utilize  known  methods  for 
solving  a  problem  even  when  the  system  provides  less  cumbersome  methods.  There  is 
also  evidence  that  more  flexible  dialogues  degrade  performance  of  relatively  inexperienced 
users. 

Complexity  is  a  measure  of  the  number  of  options  available  to  the 
user  at  a  given  point  in  the  dialogue.  Low  complexity  can  be  achieved  by  using  few  com- 
mands or  by  partitioning  the  commands  so  the  user  selects  from  a  small  set  at  any  given 
time.  [Andriole,  1989] 

Most  Windows  based  programs,  including  DSS  generators,  offer 
some  degree  of  flexibility.  A  particular  task  may  be  performed  in  a  number  of  ways  such 
as  with  a  pull  down  menu,  a  pop  up  menu  brought  up  by  double  clicking  an  item,  with  a 
control  key  sequence  or  function  key.  This  allows  the  user  to  perform  a  task  in  the  man- 
ner in  which  they  are  most  comfortable. 

b.  Input  Representation 

How  are  DSS  problems  expressed?  The  simplest  way  to  describe  a  DSS 
problem  to  a  computer  system  is  to  input  a  separate  and  explicit  description  for  each  indi- 
vidual variable  or  constraint  (or  both).  Depending  on  the  decision  problem,  various  types 
of  input  data  may  be  available  including  absolute  or  relative  data. 

In  some  instances,  the  user  may  have  absolute  data  derived  from  experi- 
ment and  testing.  In  selecting  a  new  fighter  aircraft,  the  prospective  buyer  would  have  ex- 
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plicit  knowledge  in  advance  regarding  the  speed  of  the  competing  aircraft  and  the  utility 
derived  from  various  speeds. 

Programs  which  accept  absolute  data  take  the  data  in  a  variety  of  forms  in- 
cluding: explicit  data  (e.g.  mach  1 .4)  or  as  statistical  values.  The  decision  tree  programs 
and  a  few  of  the  hierarchy  base  programs  accept  data  in  this  manner.  Where  explicit  data 
values  are  entered  in  hierarchical  programs,  the  user  also  has  the  opportunity  to  enter  util- 
ity values  for  the  various  data  points.  Many  of  the  programs  will  allow  the  user  to  enter 
statistical  data  in  place  of  actual  numbers.  This  is  useful  when  using  experimental  data  and 
results  in  output  which  represents  a  range  of  likely  outcomes. 

In  contrast,  in  selecting  a  new  car,  the  buyer  would  not  have  an  explicit 
measurement  of  his  preference  for  one  car's  style  over  another.  He  would  probably  know 
that  he  thinks  that  one  vehicle  is  attractive  while  the  other  isn't  or  he  may  know  that  he 
prefers  one  design  over  another.  In  this  case  he  has  relative  data,  he  knows  the  relative 
preference  of  one  item  over  another,  even  though  no  specific  scores  can  be  assigned.  Pro- 
grams which  use  relative  data,  require  the  user  to  choose  a  preference  of  one  item  over 
another.  This  may  be  done  by  using  a  slide  bar,  by  manipulating  icons,  or  by  entering  nu- 
merical preferences.  This  type  of  data  entry  may  be  most  useful  when  the  user  wants  to 
make  a  quick  decision  without  taking  the  time  to  gather  detailed  data. 

c  Output  Representation 

How  are  results  viewed  and  analyzed?  DSS  packages  produce  a  variety  of 
outputs  which  are  useful  in  decision  analysis.  All  programs  present  their  recommendations 
in  a  graphical  format  while  some  also  produce  tabular  data.  Graphic  displays  are  fre- 
quently much  better  communicators  of  information  than  alphanumeric. 

All  of  the  multi-criteria  based  programs  produced  a  variety  of  bar  charts 
showing  the  overall  utility  of  the  different  output  variables.  Figure  4  shows  a  stacked  bar 
chart  which  provides  details  concerning  which  variables  contributed  most  to  the  recom- 
mendation. 
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Figure  4  Multi-criteria  programs  generate  a  bar  chart  allowing  the  user 
to  evaluate  the  contribution  from  various  input  variables.  [From  LDW 
by  Logical  Decisions] 

A  few  programs  will  produce  a  greater  variety  of  outputs.  Figure  5  shows 
a  scatter  diagram  which  allows  a  quick  comparison  between  two  input  variables. 
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Figure  5  A  scatter  diagram  compares  two  variables  graphically. 
[From  LDW  by  Logical  Decisions] 

Programs  which  perform  decision  making  under  uncertainty  typically  pro- 
duce a  probability  distribution  chart  as  shown  in  Figure  6.  This  allows  the  decision  maker 
to  detennine  the  likelihood  of  the  various  outcomes. 
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Figure  6  Programs  using  decision  making  under  uncertainty  produce  prob- 
ability distribution  charts.  The  chart  on  the  left  shows  the  cumulative  dis- 
tribution. [From  DATA  by  Tree  Age] 

(1)  How  are  problems  solved?  ~  The  users  of  DSS  need  a  flexible 
mechanism  that  displays  results  quickly  and  interactively.    Whether  for  demonstration  or 
production  use,  a  DSS  must  do  more  than  simply  display  result  values  returned  by  the 
solver.  Programs  should  offer  some  degree  of  insight  into  how  the  problem  was  solved. 

Perhaps  the  most  useful  feature  offered  is  the  ability  to  perform 
sensitivity  analysis.  Sensitivity  analysis  shows  the  change  in  output  due  to  a  given  change 
in  one  or  more  input  variables.  In  the  thesis  survey  of  what  users  of  DSS,  designers,  and 
educators  felt  were  important  features  for  DSS  packages  to  have,  respondents  gave  an  av- 
erage of  30  out  of  100  points  to  sensitivity  analysis,  the  most  heavily  DSS  related 
weighted  factor.    A  typical  sensitivity  analysis  graph  is  shown  in  Figure  7. 
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Figure  7  Sensitivity  graph  shows  the  change  in  an  output  variable  when  an 
given  input  variable  is  changed.  [From  LDW  by  Logical  Decisions] 

Nearly  all  decisions  involve  situations  which  are  probabilistic  in  that 
the  behavior  of  the  system  being  modeled  cannot  be  predicted  with  certainty  because  a  de- 
gree of  randomness  is  present.  A  probabilistic  model  attempts  to  capture  the  probabilistic 
nature  of  the  system  by  requiring  probabilistic  data  inputs  and  by  generating  probabilistic 
outputs.  [Sprague,  1996]  DSS  users  assigned  30  out  of  100  available  points  to  the  im- 
portance of  the  ability  to  model  this  uncertainty.  Some  of  the  programs  reviewed  allow 
the  user  to  enter  a  variety  of  statistical  distributions  in  place  of  actual  data  values.  LDW 
for  instance  allows  the  user  to  enter  a  point  estimate,  normal  distribution,  uniform  distri- 
bution, discrete  distribution,  piecewise  linear  cumulative  or  three  point  estimate.  The  pro- 
gram will  then  show  this  uncertainty  on  the  output  as  shown  in  Figure  8. 
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Figure  8  This  chart  shows  uncertainty  as  a  range  of  possible  outputs  in  a 
decision  where  a  normal  distribution  was  assigned  to  one  of  the  decision 
criteria.  [From  Decision  Plus  by  Info  Harvest] 

(2)  Explanations  Of  Recommendations  ~  Survey  respondents  gave 
considerable  weight  to  the  importance  of  a  program  providing  insight  into  its  recommen- 
dations. According  to  the  designer  of  one  of  the  popular  packages: 

There  are  a  few  key  elements  to  trigger  the  understanding  of  a  user  to  the 
process  of  decision  making.  This  hurdle  can  be  overcome,  often  with  a  few 
minutes  of  proper  instruction  or  guidance  within  the  software.  The  trick  is 
how  to  achieve  this.    More  sophisticated  users  must  be  more  aware  of  how 
to  build  models  to  represent  a  decision  or  process.  Training  courses  are 
under  development  to  assist  both  the  basic  user  and  the  more  advanced 
user  on  how  to  fully  utilize  the  produce. 

(3)  Using  the  output  —  In  addition  to  viewing  output  on  the  screen, 
some  of  the  programs  allow  the  user  to  export  graphical  output  to  other  applications  for 
use  in  other  programs  such  as  a  word  processor    This  is  important  since  many  decision 
will  have  to  be  written  up  and  presented  to  others. 

2.         Data  Management 

It  is  increasingly  critical  for  the  DSS  to  use  all  the  important  data  sources  internal 
and  external  to  the  organization.  This  includes  concepts,  ideas  and  opinions  that  are  im- 
portant to  decision  making.  In  order  to  effectively  manage  data,  a  good  DSS  should  con- 
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tain  features  for  managing  and  manipulating  data  and  for  keeping  track  of  different  ver- 
sions of  the  data.  It  is  also  desirable  to  be  able  to  import  and  export  data  from  other  ap- 
plications such  as  databases  and  spreadsheets. 

a.  DBMS  Management  Functions  and  Version  Management 

Desirable  DBMS  functions  include  such  things  as  the  ability  to  manipulate 
rows  of  data  or  to  extract  subsets  of  a  database.  The  thesis  survey  asked  about  the  im- 
portance that  the  program  have  DBMS  functions.  Though  the  survey  respondents  allotted 
only  minimal  importance  to  this  feature,  it  is  valuable  to  have  at  least  minimal  DBMS  fea- 
tures available  if  working  with  large  data  sets. 

Version  management  allows  users  to  work  with  various  versions  of  a  data- 
base. This  is  important  when  the  data  is  being  used  by  different  users  or  being  modified 
for  different  trials.  Again,  this  feature  was  judged  relatively  unimportant  by  all  respon- 
dents but  would  be  valuable. 

b.  Data  Import  and  Export 

The  survey  asked  users  how  important  it  was  for  the  DSS  to  be  able  to  im- 
port and  export  data  directly  from  other  applications  and  databases.  Surprisingly,  corpo- 
rate users  and  program  designers  allotted  less  than  15  points  (out  of  100  available  design 
feature  points)  to  this  ability.  Educational  users  allotted  only  6  points.  This  would  lead  to 
the  conclusion  that  in  most  cases,  desktop  DSS  users  are  using  the  programs  to  model 
stand  alone  decisions  vice  repetitive  ones  which  draw  on  an  external  data  source.  In  the 
words  of  one  DSS  vendor: 
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Some  people  seem  to  use  data  import/export  extensively,  but . . .  they  are  in 
the  minority.   . . .  most  people  do  not  have  the  kind  of  data  used  by  decision 
analysis  programs  (e.g.,  probability  data)  already  prepared  and  stored  in 
different  formats.  In  most  cases,  people  seem  to  estimate,  calculate,  or  de- 
rive this  information  for  the  specific  purpose  of  building  a  decision-analytic 
model.  And  to  the  extent  that  such  data  is  being  developed  solely  for  this 
purpose,  there  is  no  reason  to  add  an  extra  "layer"  of  applications  in  order 
to  import  the  data  into  a  decision  analysis  program. 

Some  specialized  groups  of  users,  such  as  epidemiologists  (who  can  import 
probability  data  from  databases)  and  financial  analysts  (who  can  import  fi- 
nancial data  from  spreadsheets)  probably  use  import/export  features  with 
substantially  greater  frequency  than  the  overall  population  of  decision  ana- 
lysts. This  perception  is  consistent  with  the  theory  that  such  use  is  tied  to 
the  availability  of  preexisting  raw  data. 

In  spite  of  the  lack  of  emphasis,  this  is  an  important  feature.  Being  able  to 
import  and  export  data  allows  users  to  access  corporate  databases  without  having  to  re- 
enter data  into  a  DSS.  As  more  programs  offer  data  exchange  and  data  becomes  more 
widely  available,  it  is  logical  that  DSS  users  will  want  to  be  able  to  extract  data  from  cor- 
porate databases  for  use  in  DSS  programs. 

3.         Model  Management 

Models  provide  the  analysis  capabilities  for  a  DSS.  Using  a  mathematical  repre- 
sentation of  the  problem,  algorithmic  processes  are  employed  to  generate  information  to 
support  decision  making.  Among  the  more  heavily  weighted  features  of  DSS  according  to 
respondents  to  the  survey  was  the  ability  of  the  DSS  to  build  large  complex  models. 

There  are  essentially  two  schools  of  thought  regarding  the  model  function  of  deci- 
sion support  systems.  One  is  that  the  main  purpose  is  to  support  the  decision-maker  in 
whatever  style  of  decision-making  chosen.  The  other  is  that  the  support,  while  suiting  the 
decision-maker's  individual  style,  must  also  be  based  on  an  appropriate  theoretical  para- 
digm. [Edwards,  1994]  If  a  user  is  called  upon  to  defend  a  decision,  it  may  be  more  useful 
if  the  model  used  is  an  accepted  one  such  as  AHP.  In  the  words  of  a  DSS  designer: 
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It  is  important  to  be  able  to  recreate  the  "audit  trail"  that  leads  to  any  par- 
ticular utility,  so  the  math  functions  are  important.  However,  this  should 
be  very  easy  unless  the  software  uses  a  proprietary  method. 

Several  of  the  multi-criteria  based  programs  use  accepted  models  such  as  AHP  or 
SMART.  Others  use  proprietary  models. 

Model  management  features  of  a  DSS  include  features  as  being  able  to  import  and 
export  model  structure  from  other  programs  and  to  maintain  control  over  versions  of 
models. 

Versions  are  most  likely  to  proliferate  when  a  model  is  being  developed.  Even 
relatively  simple  decisions  can  easily  grow  into  models  with  hundreds  of  nodes.  Version 
management  features  allow  the  user  to  modify  a  model  and  then  return  to  a  previous  ver- 
sion if  desired.  This  is  an  important  feature,  however,  it  received  relatively  little  emphasis 
from  users  of  DSS  software. 

E.         SUPPORT 

This  section  discusses  the  methods  by  which  a  user  can  receive  help  when  learning 
to  use  a  program  and  interpret  its  output.  Help  can  come  in  a  number  of  forms  including 
paper  and  online  documentation,  sample  exercises  and  tutorials,  or  Web  based  exercises. 
Companies  offer  various  degrees  of  support  from  phone  and  email  to  full  consulting. 
User  preferences  are  discussed  in  the  next  chapter. 

1.         Vendor  Support 

What  Aspects  of  DSS  Modeling  Does  the  Vendor  Address?  While  evaluating  the 
software  available,  it  is  important  to  determine  what  the  vendors  are  offering  to  sell,  and 
to  whom.  In  many  cases  with  DSS  software  there  are  services  available.  The  software 
may  come  in  any  of  several  forms,  and  may  be  addressed  to  any  of  several  lands  of  users. 
Some  vendors  may  not  offer  a  broad  enough  range  of  services  to  meet  the  users  needs, 
while  others  may  have  a  motivation  to  sell  things  that  you  do  not  want.  All  six  of  the  ven- 
dors who  responded  to  the  survey  offer  at  least  some  consulting.  At  least  one  of  the  ma- 
jor DSS  producers  offers  primarily  consulting  with  software  being  only  a  sideline. 
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2.         Training 

Documentation  can  come  in  various  forms  including  paper  manuals,  help  files,  em- 
bedded training,  and  now,  tutorials  located  on  the  World  Wide  Web. 

Embedded  training  is  computer  based.  Far  more  than  help  routines,  embedded 
training  modules  permit  users  to  digress  for  a  moment  during  a  problem  solving  session  to 
learn  or  relearn  something  with  which  they  may  be  unfamiliar.  They  permit  users  to  learn 
as  they  go  and  even  to  decide  when  they  want  or  need  a  small  refresher  course.  Self 
paced  training  permits  users  to  learn  at  their  own  pace  while  eliminating  scheduling,  logis- 
tical and  location  problems.  [Andriole,  1989] 

F.         SUMMARY 

This  chapter  provided  an  overview  of  the  basic  features  the  potential  user  of  a  DSS 
must  consider.  Features  included  the  type  of  DSS  problem  a  particular  package  addresses 
and  the  technical  requirements  of  a  program.  DSS  packages  consist  of  three  components 
including  the  user  interface,  the  database  management  system  and  the  model  management 
system.  The  important  features  of  these  components  are  summarized  in  Table  2  below. 
Desktop  DSS  are  strongest  in  the  user  interface  area,  making  extensive  use  of  the  Win- 
dows and  mouse  interface  of  a  PC.  Most  desktop  systems  lacked  DBMS  and  MMS  fea- 
tures at  any  more  than  a  rudimentary  level. 

The  next  chapter  will  discuss  DSS  as  they  relate  to  use  in  education,  production 
and  demonstration. 
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DSS  Features 


What  Problems  Does  the  Product  Address: 

Decision  making  under  uncertainty 
Multi-criteria  decision  making 


User  Interface  Management  System 

Mouse  and  Windows  support  highly  desirable 

Clean  orderly  display  which  makes  appropriate  use  of  color 

Dialog  features  menus,  query  or  natural  language,  prompting,  or  form  filling 

Relative  versus  absolute  data  values 

Bar  chart  and  stacked  bar  chart  outputs 

Sensitivity  analysis  and  ability  to  model  uncertainty  important  and  add  insight 

Ability-  of  program  to  explain  recommendations  desirable 


Database  Management  System 

Data  import  and  export  desirable  especially  with  large  or  recurring  data  needs 

Version  management  desirable 

Model  Management  System 


Model  import  and  export  desirable 
Version  management  important 


Table  2 
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IV.      DSS  USE  IN  PRODUCTION,  EDUCATION,  AND  DEMON- 
STRATION ENVIRONMENTS 

A.  INTRODUCTION 

The  previous  chapter  discussed  some  of  the  issues  which  affect  all  users  of  deci- 
sion support  systems.  This  chapter  will  look  at  specific  users  which  have  been  classified 
into  three  categories,  those  who  use  the  programs  for  production,  those  who  use  the  pro- 
grams for  demonstration,  and  those  who  use  the  programs  in  education. 

Not  surprisingly,  there  were  differences  in  what  educators  found  important  in  DSS 
software  and  what  production  users  found  important.  We  received  survey  responses  from 
six  of  the  leading  DSS  software  designers.  One  of  the  questions  asked  of  them  was  to  al- 
locate 100  points  between  four  categories  of  users:  educational,  strategic  production,  tac- 
tical production  and  demonstration.  In  this  grouping,  designers  allocated  an  average  of  53 
points  to  strategic  use  and  24  points  to  tactical  use  while  allocating  only  1 5  points  to  the 
importance  of  educational  use.  It  is  not  surprising  that  the  responses  received  from  pro- 
duction users  of  DSS  software  were  more  closely  aligned  with  the  views  of  the  designers 
than  were  responses  received  from  educators.  Says  the  designer  of  one  program: 

Our  interests  are  not  the  same  as  the  academic  community  -  we  have  no 
theory  to  defend,  only  a  need  to  ensure  that  client  requirements  are  met  as 
best  as  possible  and  at  the  highest  technical  standard.  This  means  under- 
standing how  we  can  help  the  user  in  reaching  the  decision  through  the 
most  appropriate  mechanism. 

B.  PRODUCTION  USE 

A  growing  number  of  organizations  have  developed  DSS  for  applications  ranging 
from  production  control  to  strategic  planning.  As  discussed  in  Chapter  I,  Section  B.  1, 
decisions  can  vary  in  the  degree  of  structure  of  the  decision  making  task  which  they  sup- 
port. DSS  are  being  used  for  one  time  ad  hoc  decisions  as  well  as  recurring  institutional 
uses. 
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Desktop  DSS  are  an  ideal  tool  for  managers  faced  with  ad  hoc  decisions,  the  type 
managers  face  frequently.  The  packages  reviewed  for  this  thesis  allow  the  user  to  quickly 
set  up  decision  scenarios  and  solve  problems.  The  programs  are  also  suited  for  institu- 
tional uses,  though  they  are  somewhat  limited  by  their  scant  data  import  and  export  capa- 
bilities. 

1.  Survey  Responses 

Among  the  respondents  to  the  thesis  survey,  five  were  designers  of  DSS  software 
and  seven  were  experienced  users.  Although  a  statistically  insignificant  number,  the  inputs 
provide  insight  on  what  users  of  this  type  of  software  feel  is  important  and  serve  to  vali- 
date the  assumptions  made  for  this  thesis.  For  the  purposes  of  this  survey,  designers  of 
DSS  software  were  grouped  with  users.  Since  the  designers  interact  with  a  number  of  us- 
ers, their  answers  represent,  to  some  extent,  a  broad  view  of  what  the  typical  user  expects 
to  see  in  a  DSS.  The  survey  asked  users  of  this  type  of  software  as  well  as  designers  to 
suggest  whether  they  felt  the  programs  were  more  useful  for  strategic  use  or  tactical  use. 
In  every  instance,  the  designers  of  the  software  felt  the  programs  were  more  appropriate 
for  strategic  use  while  users  were  more  evenly  mixed.  In  the  case  of  the  users,  however, 
the  small  sample  size  likely  skewed  the  data.  The  nature  of  these  programs  lend  them 
more  towards  strategic  use.  Appendix  H  shows  detailed  responses  for  production  users 
and  designers. 

2.  Software  Selection 

Survey  recipients  were  asked  to  rank  between  six  criteria  to  consider  when  select- 
ing DSS  software  including  the  length  of  time  required  to  learn  the  program,  the  ease  of 
use,  the  availability  of  multiple  platform  support  including  use  on  a  network,  cost  and  the 
power  of  the  package.  Users,  as  well  as  designers,  ranked  the  ease  of  learning  and  using 
a  package  as  the  most  important  criteria  with  the  power  (variety  of  features,  outputs,  size 
of  models)  a  close  second.  The  cost  of  the  package  received  relatively  little  emphasis. 
The  emphasis  of  designers  can  be  summarized  in  the  words  of  one: 
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We  have  endeavored  throughout  our  history  to  offer  extremely  intuitive 
applications  that  have  enough  power  to  satisfy  the  most  sophisticated  us- 
ers. This  entails  attention  to  the  ease  of  the  user  interface,  thorough  and 
graphical  output  and  the  ability  to  interface  with  other  programs. 

3.  Application  Features 

Survey  respondents  were  asked  to  rate  between  ten  applications  related  features 
broken  down  into  user  interface,  data  handling  and  modeling  features.  The  details  of  these 
features  were  discussed  in  Chapter  H 

Users  assigned  an  average  of  approximately  30  points  to  the  importance  of  a  well 
designed  user  interface  including  mouse  and  windows  support,  a  well  laid  out  input  inter- 
face and  good  output  representations.  All  of  the  packages  reviewed  support  Windows  3.1 
though  few  specifically  support  Windows  95. 

Users  as  well  as  designers  of  DSS  gave  surprisingly  few  points  to  the  data 
handling  features  of  a  DSS.  They  allotted  the  most  points  in  this  area  to 
the  ability  to  import  and  export  data  with  little  consideration  given  to  ver- 
sion management  or  the  availability  of  DBMS  functions. 

Users  assigned  a  total  of  44  points  to  the  model  management  area  with  the  most 
points  being  allocated  to  the  importance  of  being  able  to  build  large  complex  models. 
Model  version  management  earned  nine  of  100  points  from  DSS  users.  None  of  the  pack- 
ages reviewed  incorporated  version  support  other  than  saving  files  with  different  names. 

4.  Decision  Support  Features 

The  thesis  survey  asked  production  users  to  allocate  points  between  five  DSS  re- 
lated features  including  the  ability  to  model  uncertainty,  use  subjective  data,  enter  utility 
functions,  determine  risk  and  perform  sensitivity  analysis.  The  ability  to  perform  sensitiv- 
ity analysis  was  considered  the  most  important  feature  in  this  area  followed  by  the  ability 
to  model  uncertainty.  This  only  makes  sense.  In  a  production  environment,  it  is  important 
that  a  manager  not  only  be  able  to  make  a  decision  but  also  to  understand  and  explain  the 
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decision.  Sensitivity  and  uncertainty  analysis  provide  a  valuable  tool  to  perform  this  fea- 
ture. 

5.         Support  and  Training  Features 

The  survey  asked  users  and  designers  to  rate  the  importance  of  training  functions 
including  the  availability  of  sample  exercises  and  tutorials  and  whether  the  program  pro- 
vides explanations  and  insight.  Most  respondents  felt  that  the  availability  of  sample  exer- 
cises was  very  important  giving  that  feature  26  points.  The  availability  of  tutorials  was 
just  less  important  with  18  points.  29  points  were  allocated  to  the  importance  of  the  pro- 
gram explaining  results  with  28  points  being  assigned  to  the  importance  of  the  program 
providing  insight  into  recommendations. 

Production  users  were  asked  to  rate  the  importance  of  six  support  features  includ- 
ing: an  online  list  of  frequently  asked  questions,  availability  of  discussion  groups  and  mail 
lists,  support  via  phone  or  email,  the  level  of  local  support  and  the  availability  of  online 
documentation.  The  availability  of  phone  or  email  support  was  rated  as  the  most  impor- 
tant feature.  Production  users  were  unconcerned  about  being  able  to  receive  support  via 
discussion  groups  or  mail  lists. 

C.         EDUCATIONAL  USE 

The  use  of  technology  is  changing  the  expectations  of  students  and  faculty.  Stu- 
dents who  grow  up  in  a  technological  age  will  not  accept  lectures  that  fail  to  draw  upon 
the  information  resources  on  the  Internet  and  elsewhere.  Schools  that  do  not  provide  their 
faculty  with  classrooms  where  dynamic  audio  and  visual  media  are  easily  used  will  be  un- 
able to  attract  good  faculty  and  good  students.  Like  business  and  industry,  campuses  can- 
not afford  to  ignore  technology. 

This  thesis  makes  no  special  distinction  for  "educational"  software.  In  earlier  years 
when  computer  time  was  scarce  and  interfaces  were  primitive,  college  courses  often  relied 
on  the  simplistic  interfaces  and  slow,  unreliable  algorithms  of  "student"  DSS  packages 
whose  performance  was  sufficient  for  exercises.  Today  students  can  be  taught  with  the 


36 


same  packages  that  practitioners  use  for  applications  of  realistic  size  and  complexity. 
Many  of  the  packages  reviewed  are  available  at  low  academic  prices,  with  the  least  expen- 
sive versions  being  limited  to  reasonably  large  number  of  variables  and  constraints.  In  the 
case  of  every  package  reviewed,  a  demonstration  copy  of  the  program  was  available  for 
download  on  the  World  Wide  Web. 

1.  Survey  Responses 

Among  the  respondents  to  the  thesis  survey,  14  were  users  who  taught  DSS  at 
both  the  undergraduate  and  /  or  graduate  levels.  They  used  a  variety  of  different  DSS 
packages  in  their  classes  including  spreadsheets.  The  survey  asked  educators  whether 
their  use  of  a  DSS  package  in  a  decision  support  class  was  intended  to  teach  the  student 
how  to  use  a  DSS  or  whether  it  was  used  to  support  learning  DSS  concepts.  Respondents 
were  mixed  in  their  responses  with  some  leaning  heavily  towards  teaching  DSS  use  and 
others  using  the  programs  to  support  concepts.  The  overall  response  was  nearly  evenly 
split.  The  selection  of  a  program  for  the  class  room  will  depend  in  large  part  on  its  in- 
tended purpose.  Some  programs  reviewed  follow  traditional  DSS  thinking.  For  example, 
DATA  allows  the  user  to  create  traditional  influence  diagrams  and  decision  trees.  Crite- 
rium  Decision  Plus  uses  very  traditional  AHP  diagrams.  Either  of  these  programs  would 
be  good  choices  for  use  by  a  professor  whose  main  purpose  is  reinforcing  DSS  concepts. 
Other  programs  use  less  traditional  methods.  Which  and  Why  would  be  a  good  choice  of 
programs  to  demonstrate  what  packages  are  available  but  it's  non  traditional  approach 
makes  it  less  desirable  for  teaching  concepts.  Appendix  F  provides  detailed  survey  re- 
sponses from  educators. 

2.  Software  Selection 

In  the  thesis  survey  educators  were  asked  about  five  primary  criteria  to  consider 
when  selecting  DSS  software  including  the  length  of  time  required  to  learn  the  program, 
the  ease  of  use,  the  availability  of  multiple  platform  support  including  use  on  a  network 
and  cost. 
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The  length  of  time  required  to  use  a  DSS  and  the  closely  related  criteria,  ease  of 
use,  were  ranked  as  very  important  by  respondents  to  the  survey.  These  two  features  ac- 
counted for  an  average  of  almost  60  points  out  of  100  points  allotted  the  selection  portion 
of  the  survey.  This  is  to  be  expected  given  the  relatively  short  time  which  students  have  to 
learn  and  use  a  software  package  in  the  classroom  and  the  likely  possibility  that  more  than 
one  program  will  be  taught. 

We  expected  multiple  platform  support  would  be  an  important  criteria  given  the 
variety  of  platforms  typically  used  in  schools.  Out  of  100  points  available  in  the  selection 
portion  of  the  survey,  however,  educators  allotted  only  3  points  to  cross  platform  support 
and  ten  points  to  network  use.  This  is  attributable  to  the  growing  percentage  of  WINTEL 
based  computers  and  the  decreasing  numbers  of  Apple  Macintosh. 

Not  surprisingly,  the  cost  of  acquiring  a  package  represented  31  of  the  100  points 
available  in  this  section.  The  DSS  packages  reviewed  here  range  in  price  from  $20.00  to 
over  $10,000.  In  every  case,  there  are  trial  versions  of  the  programs  evaluated  available 
for  download  from  the  World  Wide  Web,  however,  some  are  quite  limited  in  their  useful- 
ness as  instructional  tools.  In  some  cases,  model  size  is  limited,  a  small  drawback  for  edu- 
cation purposes.  In  other  cases  you  can't  save  or  print  results,  a  much  bigger  drawback. 

3.         Application  Features 

The  thesis  looked  at  a  variety  of  application  features  which  should  be  useful  to  us- 
ers in  education.    The  survey  recipients  were  asked  to  rate  between  user  interface,  data 
handling  features  and  model  features.  Each  of  these  areas  are  further  subdivided  as  dis- 
cussed below. 

Important  features  of  the  user  interface  include  mouse  and  Windows  support,  input 
characteristics  and  output  representation.    Respondents  allotted  twenty  points  to  the  ne- 
cessity of  having  mouse  and  Windows  support.  Respondents  allocated  only  9  points  to 
input  characteristics  but  gave  16  points  to  output.  The  quality  of  both  these  items  varies 
greatly  from  program  to  program. 
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Educators  gave  relatively  few  points  to  the  importance  of  a  packages  data  handling 
characteristics.  Database  management  functions  and  import  export  capabilities  each  re- 
ceived about  10  points  while  version  support  received  only  4  points.  For  the  purposes  of 
classroom  use,  it  is  unlikely  that  users  will  require  data  import  or  export  since  they  will 
likely  be  working  with  small  demonstration  data  sets.  Likewise,  it  is  unlikely  that  there 
will  be  many  versions  of  a  given  database. 

Collectively,  the  ability  to  build  a  variety  of  models  which  may  be  large  and  com- 
plex, gained  25  out  of  100  points.  Some  of  the  programs  which  model  uncertainty  will 
build  both  influence  diagrams  and  decision  trees  while  others  work  only  with  the  later. 
The  programs  which  use  multi-criteria  decision  making  use  AHP  and  in  some  instances 
SMART.    As  with  the  model  management  features,  version  support  was  of  little  impor- 
tance to  educators.  This  is  to  be  expected  as  version  support  is  more  important  in  col- 
laborative efforts  such  as  might  be  found  in  business.    The  ability  to  export  models  to 
other  applications  was  also  important,  earning  17  points.  Most  of  the  DSS  software  re- 
viewed did  a  poor  job  of  export. 

4.  Decision  Support  Features 

The  survey  asked  educators  to  allocate  points  between  five  DSS  related  features, 
the  ability  to  model  uncertainty,  use  subjective  data,  enter  utility  functions,  determine  risk 
and  perform  sensitivity  analysis.  Each  of  these  features  was  deemed  equally  important 
with  uncertainty  and  sensitivity  analysis  receiving  slightly  more  preference.  The  programs 
reviewed  perform  these  function  in  a  variety  of  ways.  Some  programs  will  allow  the  user 
to  enter  subjective  data  while  others  require  numeric  entries.    Most  of  the  programs  allow 
some  form  of  sensitivity  analysis.  Some,  such  as  LDW,  allow  the  user  to  enter  statistical 
data  where  actual  data  isn't  available  and  then  display  uncertainty  in  decisions. 

5.  Support  and  Training  Features 

Educators  were  asked  to  rate  the  importance  of  built-in  training  functions  includ- 
ing the  availability  of  sample  exercises  and  tutorials  and  whether  the  program  provides  ex- 
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planations  and  insight.  Most  respondents  felt  that  the  availability  of  sample  exercises  and 
tutorials  were  very  important  giving  those  features  27  and  33  points  respectively.  All  of 
the  programs  come  with  sample  problems.  21  points  were  allocated  to  the  importance  of 
the  program  explaining  results  with  19  points  being  assigned  to  the  importance  of  the  pro- 
gram providing  insight  into  recommendations. 

The  DSS  reviewed  offer  various  methods  of  support  for  users.  Educators  were 
asked  to  rate  the  importance  of  six  support  features  including:  an  online  list  of  frequently 
asked  questions,  availability  of  discussion  groups  and  mail  lists,  support  via  phone  or 
email,  the  level  of  local  support  and  the  availability  of  online  documentation.  The  avail- 
ability of  online  documentation  was  rated  as  the  most  important  feature.  The  other  five 
features  each  received  equal  ratings. 

D.  DEMONSTRATION 

Desktop  systems  are  quite  useful  at  the  prototyping  stage,  where  the  work  is  fo- 
cused on  building  an  acceptable  model  and  demonstrating  sufficiently  promising  results  to 
justify  a  larger  investment.  To  some  extent,  educators  also  build  demonstration  systems 
which  are  used  to  support  a  lecture  on  a  particular  type  of  decision  making. 

Depending  on  the  type  of  ultimate  model  planned,  any  of  these  programs  might  be 
useful  for  prototyping.    The  first  feature  required  is  that  the  program  support  the  type  of 
model  ultimately  planned  for  development.  The  length  of  time  required  to  construct  a 
problem  is  also  important  as  is  a  flexible  UIMS  which  can  simulate  the  full  scale  product. 
Of  less  importance  is  a  full  featured  DBMS  since  small  demonstration  data  sets  will 
probably  be  used. 

E.  VARIATIONS  ACROSS  USE  CATEGORIES 

There  were  some  significant  differences  in  what  was  important  to  educators  and  to 
production  users  as  well  as  many  similarities.  When  rating  selection  criteria,  educators 
were  most  concerned  about  the  time  it  would  take  for  their  students  to  learn  the  package 
as  well  as  the  cost  of  the  package.  This  only  makes  sense  considering  the  high  cost  of 
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some  software  packages  and  the  limited  amount  of  time  available  for  a  student  to  master  a 
program.  Production  users  not  surprisingly  were  most  interested  in  the  "power"  of  a 
packages.  They  too  were  interested  in  ease  of  use  and  learning  time,  though  less  so  than 
educators. 

1.  Application  Features 

Both  educational  users  and  production  users  assigned  roughly  the  same  number  of 
points  to  features  in  this  area.  Surprisingly,  neither  placed  much  emphasis  on  the  data 
handling  capabilities  of  the  programs,  placing  emphasis  instead  on  the  user  interface  fea- 
tures and  the  ability  to  build  large  complex  models. 

2.  DSS  Features 

There  were  some  differences  in  DSS  features  educators  and  production  users 
found  important.  Educators  allotted  three  times  as  much  emphasis  to  the  importance  of  a 
programs  ability  to  model  utility  functions.  In  a  pure  academic  setting,  this  would  be  de- 
sirable in  a  program  since  the  professor  could  interactively  demonstrate  this  important 
concept.  In  a  production  setting,  often  the  manger  may  prefer  to  simply  express  his  pref- 
erence for  one  choice  over  the  next,  leaving  the  details  of  utility  to  the  underlying  pro- 
gram. 

Production  users  ranked  the  importance  of  being  able  to  measure  uncertainty  and 
sensitivity  more  strongly  than  did  educators.  Although  educators  are  interested  in  being 
able  to  demonstrate  these  features  in  the  classroom,  production  users  must  have  this  in- 
formation to  make  informed  decisions.  Thus  the  higher  ranking. 

3.  Support  and  Training 

In  the  area  of  support  and  training,  educators  for  obvious  reasons  were  more  inter- 
ested in  online  tutorials  than  were  production  users.  Educators  were  also  more  interested 
in  being  able  to  receive  support  through  online  discussion  groups  and  mail  lists  while  pro- 
duction users  preferred  to  be  able  to  pick  up  the  phone  and  call  someone.  Both  produc- 
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tion  and  educational  users  weighted  the  importance  of  online  documentation  highly.  De- 
signers, however,  gave  this  relatively  little  priority,  apparently  preferring  to  provide  writ- 
ten documentation. 

F.         SUMMARY 

Different  DSS  features  are  more  important  to  some  users  than  others.  Designers 
of  DSS  software  not  surprisingly,  place  more  emphasis  on  the  production  use  of  their  pro- 
grams than  on  education  use.  As  such,  there  priorities  are  more  closely  aligned  with  pro- 
duction users  than  educators. 

In  a  production  setting,  DSS  are  perhaps  more  practical  for  non-recurring  strategic 
decisions.  The  limited  data  import  and  export  capabilities  of  today's  products  make  them 
less  suited  for  recurring  decisions  which  may  rely  on  external  data.  DSS  can  make  excel- 
lent supplements  to  courses  of  instruction  which  teach  DSS.  They  can  be  used  either  to 
teach  DSS  or  to  demonstrate  interactively  the  concepts  of  decision  trees  or  multi-criteria 
decision  making.  Table  3  summarizes  production  and  educational  users  priorities  when 
using  DSS.  The  following  chapter  reviews  some  of  the  leading  DSS  packages  demon- 
strating the  current  "state  of  the  art"  as  it  relates  to  the  feature  set  discussed  in  this  and  the 
previous  chapter. 


f  Judged  important 
•l  Judged  less  important 

Production  Users 

Educational  Users 

All  Users 

Selection  considera- 
tions 

TPower 
iCost 

tLength  of  time  to  learn 
tCost 

TWell  designed  UIMS 
Tjfaseof  use 
^Multiple  platform  sup- 
port 

Application  features 

t  Ability  to  build  differ- 
ent types  of  models 
tModel  Export 

t  Ability  to  build  com- 
plex models 

DSS  Features 

TSensitivity  analysis 
TModeling  uncertainty 

Training 

TExplanation  of  results 
and  insight 

TTutorials 

TSample  exercises 

Support 

fPhone  support 

TOnline  documentation 

Table  3 
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V.       DESKTOP  DESCISION  ANALYSIS  SOFTWARE 

A.  INTRODUCTION 

Having  discussed  the  feature  set  which  define  a  desktop  decision  support  system, 
this  chapter  provides  an  overview  of  currently  available  packages.  This  thesis  will  not  at- 
tempt to  review  the  packages  in  depth.  New  releases  are  being  introduced  too  fast  to 
make  such  an  assessment  of  lasting  value.  The  popular  press  contains  numerous  reviews 
of  such  packages,  some  of  which  are  listed  in  the  bibliography.  This  review  will  concen- 
trate on  how  the  currently  available  packages  meet  or  do  not  to  meet  the  requirements  set 
forth  in  the  preceding  chapters.  Table  4  shows  the  packages  reviewed  and  the  producer. 
Appendix  B  contains  a  chart  detailing  more  specific  information  on  these  programs. 


Software  Title 

Company 

@RISK,  TopRank 

Palisade  Corp. 

AliahTHINK! 

Aliah  Inc. 

Criterium  Decision  Plus 

InfoHarvest  Inc. 

Decide  Right  for  Windows 

Avantos  Performance  Systems  Inc. 

Decision  Analysis  by  TreeAge  (DATA) 

TreeAge  Software  Inc. 

Decision  Pro 

Vanguard  Software 

DPL 

Applied  Decision  Analysis 

Expert  Choice  Professional 

Expert  Choice  Inc. 

Logical  Decisions 

Logical  Decisions 

Which  &  Why 

Arlington  Software  Corp. 

Table  4 


B.         TECHNICAL  CONSIDERATIONS 

None  of  the  packages  reviewed  possessed  unusual  hardware  requirements.  A  few 
require  8  Mb's  of  RAM  but  most  will  run  with  4  Mb's.  All  packages  will  run  in  the  Win- 
dows 3  1  environment,  many  also  have  versions  designed  for  Windows  95.  Several  still 
have  DOS  versions  available.  A  few  are  available  for  Macintosh.  Technical  considera- 
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have  DOS  versions  available.  A  few  are  available  for  Macintosh.  Technical  considera- 
tions are  a  go,  no  go  situation  with  which  most  users  are  familiar.  Appendix  B  shows  a 
matrix  summarizing  the  programs  requirements. 

All  of  the  packages  listed  below  are  devoid  of  security  features  such  as  password 
protection  of  files  and  model  designs.  Even  where  intended  for  use  in  network  situations, 
there  is  no  protection  against  unintentional  or  unauthorized  modification. 

C.        USER  INTERFACE  MANAGEMENT  SYSTEM 

The  user  interface  of  the  programs  reviewed  vary  greatly  though  all  are  relatively 
easy  to  learn  and  to  use.  The  figures  in  this  section  show  some  of  the  user  interfaces  used 
to  set  up  problems  and  make  comparisons. 

There  is  considerable  difference  in  the  quality  of  the  displays  of  desktop  DSS  soft- 
ware. Some  adhere  closely  to  the  principles  of  visual  design  whereas  other  pay  little  at- 
tention. 

1.         Building  Decision  Problems 

Figures  9  through  15  show  some  of  the  variety  of  input  methods  offered  by  the 
packages  reviewed.  For  the  most  part,  desktop  DSS  offer  the  user  few  options  for  con- 
structing decision  problems.  Most  make  use  of  Windows  pull-down  menus  and  toolbox 
buttons  which  allow  the  user  to  "construct"  a  problem  on  a  clean  screen.  A  few  of  the 
programs  allow  the  user  to  start  with  a  "brainstorming"  feature.  This  feature  allows  the 
user  to  list  random  thoughts  in  a  "note  pad"  area  of  the  screen,  then  arrange  these 
thoughts  hierarchically  and  finally  convert  the  list  to  a  decision  matrix. 
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Figure  9  Criterium  Decision  Plus  uses  a  traditional  AHP  graph  to  de- 
sign the  model.  Values  can  be  entered  using  either  pairwise  compari- 
son or  absolute  scores.  The  user  can  select  either  AHP  or  SMART  as 
the  analysis  method.  [From  Decision  Plus  by  Info  Harvest] 
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Figure  10  Which  &  Why  uses  slide  bars  to  set  up  a  pairwise  comparison. 
Once  decision  criteria  are  prioritized,  the  user  selects  subjectively  between 
the  different  outcome  variables  as  they  relate  to  each  criteria.  A  difficulty 
with  this  method  is  that  if  the  user  is  weighing  a  number  of  alternatives,  he 
must  remember  the  score  assigned  to  a  given  data  value.  [From  Which  and 
Why  by  Arlington  Software] 
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Figure  1 1  Expert  Choice  uses  a  matrix  based  on  AHP  to  set  up  the  decision 
model.  [From  Expert  Choice  by  Expert  Choice] 
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Figure  12  Once  the  decision  model  is  set  up  in  Expert  Choice,  users  can 
chose  several  methods  of  weighing  alternatives,  a  matrix  based  method 
shown  on  the  left,  or  a  graphical  method,  on  the  right.  [From  Expert 
Choice  by  Expert  Choice] 
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Figure  13  Decide  Right  uses  a  less  traditional  methodology.  In  ranking 
criteria,  the  user  drags  boxes  corresponding  to  the  different  criteria  to- 
wards the  top  left  as  they  become  more  important.  [From  Decide  Right  by 
Avantos  Performance  Systems] 
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Figure  14  DATA  uses  influence  diagrams  to  formulate  the  decision 
model.  Once  the  diagram  is  drawn,  the  user  then  uses  pop  up  boxes  to 
enter  probabilities  and  values  about  the  various  nodes    The  influence 
diagram  may  then  be  converted  into  a  decision  tree.  DPL  works  in  a 
similar  manner  [From  DATA  by  Tree  Age] 
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Figure  15  In  Decision  Pro  the  users  starts  with  the  decision  tree  view.  Us- 
ers build  the  tree  node  by  node,  entering  data  values  and  probabilities  as 
they  go.  [From  Decision  Pro  by  Vanguard  Software] 


2.         Outputs 

Outputs  from  the  programs  have  more  in  common  than  the  overall  interfaces.  The 
basic  output  of  all  of  the  AHP  based  programs  is  a  bar  chart  showing  a  ranking  of  outputs 
as  shown  in  Figure  16.  Most  programs  also  offer  a  stacked  bar  chart  which  shows  the 
contribution  from  each  of  the  criteria.  Figure  17  shows  this  type  of  chart.  Decide  Right 
will  print  a  "plain  English"  decision  report  and  export  it  to  a  word  processor  for  editing. 
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Figure  16  Criterium  Decision  Plus  and  most  programs  provide  a  bar  chart 
which  shows  overall  score  of  each  output  variable.  [From  Decision  Plus  by 
InfoHarvest] 
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Figure  17  Several  of  the  programs  provide  a  breakdown  of  which  criteria 
contributed  most  to  the  output.  This  allows  the  decision  maker  to  deter- 
mine which  criteria  contribute  most  to  the  various  outcomes.  [From  LD W 
by  Logical  Decisions] 
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Programs  offer  a  variety  of  ways  to  measure  sensitivity,  an  essential  ingredient  of  a 
good  DSS.  Figure  7  shows  the  traditional  sensitivity  chart.  Some  programs  offer  the 
useful  feature  of  dynamic  sensitivity  as  shown  in  Figurel8. 
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Figure  1 8  Some  programs  offer  dynamic  sensitivity  analysis.  In  this  pro- 
gram, as  the  user  adjusts  the  criteria  weights  (shown  on  the  left)  using  the 
mouse,  the  output  recommendations  (on  the  right)  adjust  accordingly.  This 
allows  useful  insight  into  the  importance  of  different  input  variables.  [From 
Expert  Choice  by  Expert  Choice] 

Scatter  diagrams,  Figure  19,  and  spider  (or  Radar)  diagrams,  Figure  20,  provide 
the  user  additional  ways  of  reviewing  data. 
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Figure  19  Some  multi-criteria  programs  produce  a  scatter  diagram  which 
allow  the  decision  maker  to  compare  two  variables  for  each  alternative. 
[From  Expert  Choice  by  Expert  Choice] 
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Figure  20  One  program  produces  a  Spider  (or  Radar)  chart  which  shows 
weights  assigned  to  different  criteria.  [From  AliahThink  by  Aliah] 

Decision  tree  based  programs  such  as  DATA  or  Decision  Pro  produce  risk  profile 
graphs  as  shown  in  Figure  21.  This  graph  shows  the  user  the  range  and  likelihood  of  out- 
comes. 
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Figure  21  Programs  which  use  decision  making  under  uncertainty  provide 
risk  profile  graphs.  These  graphs  show  the  predicted  likelihood  of  the 
various  possible  outcomes.  [From  Decision  Pro  by  Vanguard] 


3.         Macros  and  Programming 

With  the  exception  of  the  spreadsheet  add-ins,  none  of  the  programs  provide  for 
macros  though  AliahThink  does  offer  a  form  of  programming  which  allows  the  user  to 
write  customized  routines.  All  of  the  programs  allow  models  to  be  built  and  saved  and 
then  run  by  other  user  thus  allowing  a  sort  of  "expert  knowledge  base."  None  of  these 
programs  work  well  with  automated  routines  such  as  a  user  might  write  with  Delphi  or 
Visual  Basic  though  an  experienced  programmer  could  force  the  programs  to  work  with 
considerable  effort. 


D. 


DATA  MANAGEMENT  SYSTEM 


The  programs  reviewed  offer  fairly  weak  DBMS  features.  In  most  cases,  once 
data  is  entered  the  user  is  unable  to  perform  features  such  as  search  and  query  or  to  ex- 
tract subsets  of  the  data.  None  of  the  programs  offer  version  support.  As  an  alternative, 
however,  all  programs  allow  the  user  to  use  the  "save  as"  feature  to  save  different  data 
sets  and  rankings. 
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As  discussed  in  Chapter  HI,  Section  D  3b.,  a  key  factor  to  consider  when  select- 
ing the  system  is  the  type  of  data  which  will  be  used.  If  the  decision  problem  being  solved 
uses  absolute  vice  relative  values,  the  choice  of  programs  is  limited  to  those  which  will  ac- 
cept actual  data.  LDW  and  Decision  Plus  are  examples  of  programs  which  allow  the  user 
to  enter  numerical  data  and  assign  utility  functions.  Other  programs,  such  as  Which  and 
Why  and  Decide  Right  force  the  user  to  make  pairwise  comparisons  between  the  different 
decision  variables.  This  method  essentially  forces  the  user  to  determine  a  utility  function 
value.    Not  being  able  to  enter  data  is  a  limitation  in  some  cases,  however,  for  a  simple 
decision  where  the  user  doesn't  wish  to  spend  time  gathering  data,  a  subjective  compari- 
son may  suffice.  For  example,  if  ranking  the  desirability  of  the  fighter  aircraft  with  a  top 
speed  of  mach  1.5,  the  user  might  choose  a  score  of  eight  often  or  nine  often  where  as  a 
plane  with  a  speed  of  mach  1 .4  might  be  assigned  a  score  of  seven  or  eight.    A  significant 
draw  back  to  this  method  is  that  when  entering  large  amounts  of  data,  the  user  might  be 
inconsistent  in  assigning  scores. 

Some  of  the  programs  offer  data  import  and  export  capabilities.  None  perform  the 
task  in  a  completely  straight  forward  manner,  however,  most  will  create  a  comma  delim- 
ited file  and  some  will  save  data  in  other  formats  such  as  Microsoft  Excel  or  Lotus  123. 
Table  5  shows  a  sample  export  table  from  Expert  Choice.  Numbers  indicate  the  corre- 
sponding utility  value  of  each  criteria.  Table  6  shows  a  similar  output  produced  by  LDW. 
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Car 

PRICE 

0.16447 

GRAND  AM 

0.04785 

MAXIMA 

0.02784 

MERCEDES 

0.00899 

VOLVO 

0.01022 

THUNDER 

0.06957 

Table  5  Sample  export  table  showing  utility  values  assigned  to  price. 
[From  Expert  Choice  by  Expert  Choice] 


ALTERNATIVES 

NAME 

Price 

Power 

Fuel  Economy 

Styling 

Chevy  S-10 

10000 

140 

16.5 

Ugly 

Mitsubishi 

11000 

109 

21.25 

Mediocre 

Toyota 

15000 

125 

21 

Mediocre 

Ford  Ranger 

14000 

160 

17.5 

Attractive 

Dodge  Ram-50 

9000 

109 

23 

Muscular 

Dodge  Dakota 

17500 

175 

15.5 

Nondescript 

NOMORE 

ALTERNATIVE  UnLITIES 

NAME 

Best  Truck 

Cost 

Performance 

Styling 

Chevy  S-10 

0.433423794 

0.72134414 

0.055692126 

5 

Mitsubishi 

0.715351046 

0.824600757 

0.600205822 

3.848591549 

Toyota 

0.465027273 

0.3 

0.721473789 

Ford  Ranger 

0.420115691 

0.322454794 

0.521369844 

1 

Dodge  Ram-50 

0.691738966 

0.733333435 

0.61926907 

2 

Dodge  Dakota 

0.250020476 

0.148733611 

0.392523047 

3 

NOMORE 

E. 


Table  6  Sample  export  tables  showing  data  entered  into  program  (top)  and 
utility  scores  (bottom).  Unfortunately,  these  tables  contain  only  numeric 
data,  not  the  formulas  which  were  used  to  calculate  the  numbers.  [From 
LDW  by  Logical  Decisions] 

MODEL  MANAGEMENT  SYSTEM 


The  model  management  system  of  the  DSS  reviewed  are  fairly  rigid  which  is  ex- 
pected of  a  DSS  generator.  The  programs  take  different  approaches  to  solving  decision 
problems.  Among  the  programs  which  solve  decisions  under  uncertainty,  all  use  decision 
trees  to  solve  the  problem.  DATA  and  DPL  offer  the  user  the  choice  of  starting  with  an 
influence  diagram.  Decision  Pro  starts  with  the  decision  tree.  Of  the  programs  which 
solve  multi-criteria  decisions,  the  programs  fall  into  two  broad  categories,  those  that  use 
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fairly  traditional  methods  such  as  AHP  or  SMART  and  those  which  use  unique  decision 
models.  LDW  or  Decision  Plus  are  examples  of  programs  which  use  traditional  decision 
analysis  models  while  Decide  Right  and  Which  and  Why  use  their  own  models.  None  of 
the  programs  offer  you  the  ability  to  export  models  to  other  applications.  Some  of  the 
programs  offer  a  form  of  version  support.  DATA  and  Decide  Right  for  instance,  allow 
the  user  to  save  different  versions  of  a  model  and  then  recall  them  later.  In  most  program 
the  user  must  use  the  "save  as"  command  to  save  different  versions  of  the  model  as  differ- 
ent file  names. 

F.  GROUP  FEATURES 

Though  the  programs  reviewed  for  this  thesis  are  not  categorized  as  group  deci- 
sion support  systems,  a  few  do  offer  group  DSS  features.  Which  and  Why  allows  multiple 
users  to  rank  their  preferences  and  combines  these  preferences  into  a  group  decision. 
Different  weights  can  be  assigned  to  different  decision  makers.  Decide  Right  allows  the 
user  to  enter  different  scenarios  which  can  be  compared  graphically.  Group  features  may 
be  desirable,  especially  in  higher  level  decision  making  where  decisions  are  frequently 
made  by  group  consensus. 

G.  SUPPORT 

Program  support  and  instruction  are  provided  in  several  manners  including  help 
files,  tutorials,  documentation,  and  various  modes  of  operation.  All  of  the  programs  come 
with  sample  problems  included  in  the  documentation. 

Online  help  varies  from  program  to  program.  Where  as  some  offer  only  basic  in- 
formation about  the  main  features  of  the  program,  some  offer  a  virtual  course  in  DSS. 
Criterium  Decision  Plus,  for  instance,  discusses  in  detail  AHP  and  SMART,  the  decision 
making  process  and  how  to  analyze  output.  Decision  Pro  offers  a  similar  level  of  detail 
for  problems  involving  decision  making  under  uncertainty.  Others  programs,  such  as 
LDW  or  DATA,  offer  only  basic  information  about  the  program  features.  A  very  useful 
feature  found  in  some  programs  is  the  presence  of  a  HELP  buttons  on  individual  windows 
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as  shown  in  Figure  22  This  allows  the  user  to  step  back  for  a  moment  and  refresh  their 
knowledge  on  a  given  feature. 
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Figure  22  The  figure  on  the  left  has  a  help  button  on  each  window  in  case 
the  user  wants  more  help  on  the  given  feature.  The  figure  on  the  right 
doesn't.  [From  Decision  Plus  by  Info  Harvest  and  LD W  by  Logical  Deci- 
sions] 

Some  of  the  programs  have  online  computer  tutorials,  and  all  come  with  sample 
problems  which  the  user  can  work  through.  Expert  Choice's  tutorial  includes  onscreen 
movies  which  show  the  novice  user  how  to  use  the  program.  Some  have  tutorials  which 
can  be  run  on  the  World  Wide  Web. 

None  of  the  programs  offer  different  expert  and  novice  modes  of  operations 
though  in  Decide  Right,  the  user  can  "turn  off"  pop  up  screens  which  offer  instructions 
once  they  are  familiar  with  the  program.  Notwithstanding,  many  of  the  programs  offer  the 
user  different  methods  of  accomplishing  the  same  task  such  as  pull  down  menus  for  the  in- 
experienced user  to  control-key  functions  for  the  more  experienced. 

Most  of  the  vendors  provide  some  degree  of  consulting  services  and  on  sight 
training  and  decision  coaching.  Vendor  Web  sites  discuss  the  level  of  services  available. 
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H.         SPREADSHEET  ADD-INS 

Spreadsheet  add-ins  offer  an  alternative  method  of  conducting  analysis.  Though 
not  traditional  DSS,  these  programs  bring  a  greater  degree  of  analysis  capability  to  a  pro- 
gram, the  spreadsheet,  with  which  many  are  already  familiar. 

The  techniques  of  risk  analysis  provide  powerful  tools  to  decision  makers  in  man- 
aging decisions  subject  to  uncertainty.  Using  the  traditional  spreadsheet,  users  can  enter 
single  point  estimates  of  variables  and  predict  a  single  output.  By  manipulating  the  vari- 
ables, the  users  can  look  at  a  range  of  outputs,  however,  with  multiple  variables  this  can 
quickly  become  a  burdensome  task.  A  limited  number  of  trials  with  varied  data,  or  a  non 
systematic  approach  can  lead  to  an  output  which  provides  little  insight.  Spreadsheet  add- 
ins  attempt  to  solve  this  problem    These  programs  allow  users  to  vary  input  to  perform  a 
number  of  simulations  and  to  show  a  range  of  potential  outcomes.  Input  can  be  varied  in 
a  variety  of  methods  including  multiple  point  estimates  and  over  20  statistical  distribu- 
tions. The  output  can  then  be  viewed  in  a  number  of  ways  including  distribution  graphs, 
sensitivity  graphs,  and  text  output. 

Perhaps  the  biggest  advantage  of  these  programs  is  that  they  run  inside  a  spread- 
sheet thus  making  full  use  of  the  spreadsheets  powerful  interface  and  programming  capa- 
bilities. All  spreadsheet  features  are  available  including  data  import  and  export,  object 
embedding  and  linking  (OLE),  macros,  and  more.  What  is  perhaps  most  important  is  that 
most  users  are  familiar  with  spreadsheets  eliminating  the  need  to  learn  to  use  an  entirely 
new  package.  Figures  23  through  26  illustrate  the  features  of  the  spreadsheet  add-in. 
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Figure  23  The  user  starts  in  the  traditional  spreadsheet.  In  this  example  a 
company  is  starting  a  new  product  line.  The  bottom  line  figure  is  the  net 
present  value  of  the  cash  flow  over  ten  years.  The  initial  price  of  the  new 
product  and  the  start  up  costs  will  be  varied  using  various  statistical  distri- 
butions. Note  the  added  toolbar  buttons.  The  user  selects  fields  to  vary 
(shown  in  red)  and  then  starts  the  simulation.  [From  @Risk  by  Palisade] 
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W  Simulation  Data 
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Figure  24  The  program  produces  a  variety  of  statistics  for  the  resulting 
output.  [From  @Risk  by  Palisade] 
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Figure  25  This  graph  shows  the  probability  distribution  for  the  net  present 
value  in  this  problem.  This  give  the  decision  maker  much  greater  insight 
than  a  single  point  estimate.  [From  @Risk  by  Palisades] 
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.JF  Summary  Graph  -  Celts  CS1  toL3l 
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Figure  26  This  graph  shows  the  potential  range  of  values  of  cash  flow  over 
the  subsequent  ten  year  period  [From  @Risk  by  Palisade] 
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VI.      CONCLUSIONS  AND  RECOMMENDATIONS 

A.  CONCLUSIONS 

The  primary  focus  of  the  software  vendors  continues  to  be  on  analysis  features 
rather  than  support  for  non-analysts  in  structuring  the  problem.  These  analysis  features 
are  useful  to  all  users  and  critical  for  the  insight  that  decision  makers  need.  However,  the 
result  is  that  there  are  relatively  few  people  that  are  qualified  and  comfortable  in  using 
these  packages.  These  packages  will  never  become  as  widespread  as  spreadsheets  and 
databases  as  long  as  the  user  has  to  be  a  highly  educated  analyst.  [Booed,  1996] 

Users  currently  have  the  choice  of  two  levels  of  software,  those  designed  for 
someone  with  experience  in  DSS  and  those  with  less  experience.  Programs  such  as  LDW 
and  Decision  Plus,  as  well  as  the  decision  making  under  uncertainty  based  programs,  as- 
sume that  the  user  has  considerable  knowledge  in  decision  methods.  Other  programs, 
such  as  Which  &  Why  and  Decide  Right,  use  a  less  traditional  techniques  for  reaching  de- 
cisions. These  programs,  however,  require  little  DSS  knowledge  from  the  user  and  essen- 
tially guide  the  decision  maker  through  making  the  decision  relying  on  intuitive  interfaces. 
The  type  of  software  most  appropriate  in  a  given  situation  depends  on  the  users  needs. 

There  are  a  variety  of  excellent  programs  available  which  offer  differing  ap- 
proaches to  decision  making.  As  discussed  in  this  thesis,  a  careful  match  between  the  us- 
ers needs  and  the  features  offered  by  these  program  should  provide  a  product  which  will 
enhance  the  decision  making  or  educational  process. 

B.  RECOMMENDATIONS 

This  thesis  touched  only  briefly  on  group  support  features  which  are  becoming 
much  more  common.  Further  research  on  these  program  in  the  format  of  this  thesis  would 
be  beneficial.    This  thesis  did  not  focus  on  real  world  scenarios  where  use  of  a  desktop 
DSS  generator  would  be  beneficial.  Research  involving  the  practical  use  of  these  power- 
ful programs  in  DoD  would  be  useful. 
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APPENDIX  A  -  METHODS  FOR  DECISION  MAKING 

There  are  a  number  of  techniques  which  have  been  developed  to  assist  the  manager 
in  making  decisions.  Though  we  typically  solve  these  problems  with  the  aid  of  a  computer 
program,  they  do  not  necessarily  require  the  use  of  a  computer.  These  models  serve  not 
only  to  present  an  answer  to  a  problem  but  to  allow  the  decision  maker  to  develop  an  in- 
sight into  the  problem  itself  by  examining  various  facets  of  it  and  by  exploring  alternative 
ways  of  looking  at  the  problem. 

When  using  decision  analysis,  a  problem  is  broken  down  into  clearly  defined  com- 
ponents which  depict  all  options,  outcomes,  inputs  and  probabilities.  These  values  are 
quantified  thus  permitting  the  evaluation  of  costs  and  benefits  associated  with  alternative 
courses  of  action.  This  process  does  not  replace  the  decision  maker  with  arithmetic, 
rather,  it  provides  an  orderly  and  more  easily  understood  structure  that  helps  to  support 
the  decision  maker  by  providing  him  with  logically  sound  techniques  to  support  and  ensure 
internal  consistency  of  his  judgments. 

There  are  several  popular  models  in  use  in  decision  analysis  including  the  decision 
tree,  influence  diagram  and  the  analytical  hierarchy  process. 

A.  INFLUENCE  DIAGRAMS 

Among  the  most  basic  decision  analysis  tools  is  the  influence  diagram.  Influence 
diagrams  consist  of  blocks  which  represent  decisions,  chances  and  outcomes  and  the  rela- 
tionship which  exists  between  these  events.  Figure  27  shows  a  decision  problem  con- 
cerning how  to  invest.  The  influence  diagram  does  not  solve  the  problem  but  allows  the 
decision  maker  to  visualize  the  decision  and  the  factors  which  influence  the  outcome. 
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Figure  27  DATA  uses  influence  diagrams  to  allow  the  decision 
maker  to  formulate  the  decision  problem.  In  the  diagram,  blue 
squares  represent  decisions,  green  circles  represent  chance  oc- 
currences and  the  red  diamond  represents  the  outcome.  [From 
Decision  Analysis  by  Tree  Age] 


B.         DECISION  TREES 

Decision  trees  offer  a  way  to  structure  and  represent  a  problem  and  choose  the 
course  of  action  consistent  with  certain  kinds  of  objectives.  In  order  to  use  a  decision 
tree,  the  decision  maker  must  know,  or  be  able  to  estimate  the  possible  actions  or  choices, 
the  cost  or  value  of  each  action,  and  how  likely  each  action  is.  Given  this  information,  the 
decision  tree  returns  an  expected  value  of  each  path  from  which  the  decision  maker  will 
chose  the  best  path. 

Decision  trees  consist  of  two  types  of  nodes,  chance  and  choice.  Chance  nodes 
represent  events  outside  the  decision  makers  control  and  involve  a  percentage  likelihood 
that  the  given  event  will  occur.  Choice  nodes  represent  decisions  which  must  be  made. 
Figure  28  shows  a  decision  tree  which  depicts  the  investment  decision  shown  as  an  influ- 
ence diagram  in  Figure  27.  In  this  scenario,  outcomes  (profit)  are  listed  on  the  right.  De- 
cision nodes  are  shown  as  blue  boxes.  There  are  two  choices  for  the  decision  maker,  to 
invest  in  CD's  paying  5%  or  in  a  risky  investment.  Chance  nodes  are  shown  as  green  cir- 
cles with  the  percentage  likelihood  of  each  eventuality  shown,  as  well  as  the  expected 
value  of  the  overall  node.  The  preferred  path  based  on  expected  value  is  shown  in  ma- 
genta. 
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Figure  28  This  decision  tree  was  generated  by  DATA  and  shows  a  decision 
to  drill  for  oil.  [From  Decision  Analysis  by  Tree  Age] 

Decision  trees  represent  problems  at  an  elemental  detail  level.  Problem  can 
quickly  grow  to  hundreds  of  nodes. 


C. 


MULTI-CRITERIA  DECISION  MAKING 


Multiple-criteria  decision  making  is  applicable  to  decision  problems  with  a  decision 
maker  considering  multiple  alternatives  whose  objective  function  values  are  known  with 
certainty.  Consider  the  following  example.  You  are  choosing  between  two  fighter  air- 
craft, the  details  of  which  are  shown  in  Table  7. 


Aircraft  A                   Aircraft  B 

Price 

SDeed 
Range 

$17  million                 $19  million 
Machl.7                   Machl.8 
800  miles                    700  miles 

Table  7 

In  this  example,  the  decision  maker  must  decide  if  aircraft  A,  which  has  a  lower 
price  but  lower  performance  figures,  is  better  than  aircraft  B,  a  higher  performance  but 
more  expensive  aircraft.  The  decision  support  system  must  transform  the  scores  of  the 
different  criteria  to  comparable  values  or  scales  which  may  or  may  not  be  linear. 

There  are  several  issues  to  consider.  In  this  example,  the  criteria  are  on  different 
scales.  Is  a  speed  of  mach  1.7  and  a  range  of  800  miles  better  or  worse  than  a  speed  of 
mach  1.8  and  a  range  of  700  miles?  In  order  to  evaluate  this  situation,  we  need  to  trans- 
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form  these  scores  to  comparable  values  or  scales.  Another  problem  arises  from  the  fact 
that  even  for  individual  criteria,  the  utility  functions  may  not  be  linear.  Is  the  100  mile 
difference  between  a  range  of  700  and  800  miles  as  important  as  the  difference  between 
300  and  400  miles?  In  order  to  resolve  these  issues,  it  is  necessary  assign  new  scales  of 
measurement  using  some  upper  and  lower  bound  and  then  evaluate  the  weighted  scores 
for  each  alternative. 

D.         ANALYTICAL  HIERARCHY  PROCESS  (AHP) 

The  analytical  hierarchy  process  is  a  theory  of  measurement  for  dealing  with  deci- 
sion problems  characterized  by  multiple  objectives,  criteria  and  multiple  alternatives.  The 
AHP  method  focuses  on  preparing  a  hierarchical  design  and  conducting  an  evaluation 
which  emphasizes  the  expertise  of  the  decision  maker  given  a  set  of  alternatives.  In  the 
design  phase,  the  decision  maker  breaks  the  problem  into  constituent  parts  and  develops  a 
hierarchy  of  objectives,  criteria  and  alternatives.    In  the  aircraft  example  the  objective  is  to 
procure  the  best  aircraft  for  the  dollar.  The  criteria  include  speed,  range  and  a  host  of 
other  features  not  shown.  Our  alternatives  include  aircraft  A  and  B.    In  the  evaluation 
phase,  the  decision  maker  rates  the  alternatives  on  each  criteria  and  develops  relative 
weights  using  pairwise  comparison.  Each  element  in  a  level  is  compared  in  relative  terms 
for  each  criteria  just  above  the  element.  The  output  from  this  is  a  relative  scale  of  meas- 
urements of  the  priorities  or  weights  of  elements.  Figure  29  29  shows  one  program's  ap- 
proach to  developing  a  solution  to  our  aircraft  problem  using  AHP. 

There  are  several  considerations  involved  in  using  AHP  including  what  kind  of  hi- 
erarchy should  be  used  and  what  should  go  into  each  level.  The  designer  must  consider 
the  scale  and  measurements  to  use.  Inconsistencies  can  develop  due  to  the  use  of  relative 
scales  or  absolute  measurements  (e.g.,  good,  average,  poor). 
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Figure  29  Criterium  Decision  Plus  utilizes  AHP  in  decision  making.  [From 
Decision  Plus  by  Criterium] 


E.  SIMPLE  MULTIATTRIBUTE  UTILITY  TECHNIQUE  (SMART) 

Another  ranking  technique  is  the  Simple  Multiattribute  Utility  Technique 
(SMART)  which  is  available  in  Decision  Plus  and  LDW. 

Before  the  user  can  synthesize  the  contributions  of  criteria  with  differing  scale,  the 
decision  model  must  provide  a  method  that  allows  the  user  to  handle  model  scales  on  an 
equal  footing.  For  example,  in  the  aircraft  selection  problem  above,  the  decision  maker 
needs  to  assign  comparable  scales  to  speed  and  range.  In  both  AHP  and  SMART,  the 
model  technique  itself  handles  this  problem.  This  is  achieved  by  normalization,  where  all 
scales  are  converted  to  a  common  internal  scale  that  takes  a  value  between  0  and  1 . 

In  AHP,  to  get  the  effective  importance  of  a  subcriterion,  the  program  takes  its 
user-given  score  divided  by  the  sum  of  the  scores  of  all  the  other  subcriteria  of  the  com- 
mon criterion.  This  guarantees  that  no  matter  what  the  original  score,  all  weights  will  fall 
between  0  and  1 . 

This  is  a  very  simple  and  effective  method  that  allows  integration  of  units  that  are 
completely  different.  However,  it  is  clearly  a  relative  judgment  method.  How  a  particular 
alternative  scores  in  the  model  depends  on  what  other  alternatives  are  being  considered. 
For  instance,  in  AHP  if  you  created  a  model  for  one  set  of  alternatives,  adding  a  new  al- 
ternative later  will  change  the  scores  of  the  original  alternatives,  sometimes  in  surprising 
ways. 
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SMART  does  not  use  a  relative  method  for  scaling  units  to  a  standard  scale  (from 
0  to  1).  Instead,  the  user  can  define  their  own  method  for  doing  this  using  a  value  func- 
tion. A  value  function  allows  decision  makers  to  explicitly  define  how  each  value  on  the 
scale  is  transformed  to  the  common  model  scale. 


F. 


UTILITY  THEORY 


Utility  theory  offers  an  approach  for  making  and  understanding  decisions.  It  is 
based  on  a  subjective  view  of  how  valuable  certain  payoffs  are  when  outcomes  are  either 
certain  or  probabilistic  and  is  a  measure  of  the  worth  one  attaches  to  an  outcome  or  situa- 
tion. Utility  is  subjective  and  may  change  over  time.  Rational  decision  makers  will  at- 
tempt to  maximize  utility.  Utility  functions  are  frequently  expressed  graphically.  Figure 
30  shows  the  utility  value  for  given  speeds  in  our  aircraft  problem. 
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Figure  30  Criterium  uses  a  graphical  approach  to  assigning  utility  value. 
The  user  uses  the  mouse  to  adjust  the  curve  to  the  appropriate  utility  value. 
[From  Decision  Plus  by  Criterium] 

In  this  example,  utility  value  remains  relatively  low  until  the  plane's  speed  reaches 
mach  one.  A  fighter  aircraft  with  a  speed  below  mach  one  has  little  utility  regardless  if  the 
speed  is  mach  .  1  or  mach  .5.  The  utility  increases  rapidly  up  until  about  mach  1.5.  Above 
this,  little  additional  utility  need  be  assigned  because  the  plane  is  already  faster  than  most 
of  our  competitors.    By  converting  a  raw  data  value,  mach  0  to  mach  2,  to  a  utility  score 
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of  zero  to  one,  we  can  now  use  this  number  can  now  be  used  in  comparison  with  other 
factors  involved  in  our  decision. 

G.         WEIGHTING  AND  UTILITY  FUNCTIONS 

All  DSS  allow  the  decision  maker  to  assign  weights  to  the  criteria  being  measured. 
Most  use  a  graphical  approach  utilizing  some  form  of  pairwise  comparison  which  allows 
the  user  to  rank  their  preference  of  criteria  over  another.  Figure  3 1  shows  one  example 
where  the  user  uses  slide  bars  to  rank  each  feature  on  a  ten  point  scale. 
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Figure  3 1  Which  and  Why  uses  a  graphical  method  to  assign  values  to 
preferences.  In  this  example,  the  user  uses  slide  bars  to  rank  his  pref- 
erence for  one  feature  over  another.  The  pie  chart  shows  the  overall 
preference  weights.  [From  Which  and  Why  by  Arlington  Software] 

MONTE  CARLO  SIMULATION 


Monte  Carlo  simulation  is  used  for  modeling  uncertainty  to  help  manage  risk  and 
simulate  complex  systems.  Risk  is  generally  thought  of  as  either  the  chance  of  some  un- 
fortunate event  happening  or  as  the  volatility  of  a  key  performance  measure  such  as  prof- 
its. Decision  trees  provide  an  excellent  method  of  planning  uncertain  actions  to  take  in  re- 
sponse to  events.  Monte  Carlo  simulations,  on  the  other  hand,  are  best  for  modeling  un- 
certainty and  volatility. 

Monte  Carlo  simulation  allows  the  user  to  replace  uncertain  quantities  in  the  model 
with  the  results  of  a  number  of  random  trials  and  then  see  how  that  uncertainty  affects  the 
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results.  Like  decision  trees,  Monte  Carlo  simulations  result  in  an  expected  value  that  aids 
in  choosing  the  most  attractive  course  of  action.  They  also  provide  information  about  the 
range  of  outcomes  such  as  best-  and  worst-case,  probability  of  reaching  specific  targets, 
most  likely  outcomes,  etc. 

One  advantage  of  Monte  Carlo  simulation  is  that  it  is  easy  to  apply.  When  com- 
bining several  uncertain  values,  deterniining  the  uncertainty  on  the  result  can  be  very  com- 
plex. For  example,  when  adding  two  uncertain  values,  the  uncertainty  on  the  result  is 
somewhat  less  that  the  sum  of  the  original  uncertainties.  Using  Monte  Carlo  simulation, 
this  and  similar  effects  are  handled  automatically  so  the  user  doesn't  need  to  know  much 
about  statistics  to  get  accurate  results. 
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@nalyst  provides  so- 
phisticated analytics 
functions  for  bonds, 
options,  mbs,  business 
dates  and  amortiza- 
tion. Also  available  as 
c-subroutine  libraries. 

@Risk  is  the  Risk 
Analysis  add-in  for  Ex- 
cel and  Lotus  1-2-3. 
@Risk  adds  the  power 
of  Monte  Carlo  simu- 
lation to  any  spread- 
sheet model.  Gener- 
ates graphs  and  de- 
tailed statistical  reports 

AXA  solves  linear  and 
integer  linear  pro- 
gramming models.  The 
AXA  engine  is  a  dy- 
namic link  library 
available  for  use  by 
other  programming 
languages  like  Visual 
Basic,  C++  and  For- 
tran. 

BestFit®  is  distribution 
fitting  solution  for  Win- 
dows. BestFit  finds  the 
statistical  distribution 
that  best  fits  any  data 
set  up  to  30,000 
points.  Results  can  be 
used  in  @Risk. 

Crystal  Ball  is  a  lead- 
ing Monte  Carlo  simu- 
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lation  add-in  for  Excel 
and  Lotus  1-2  that 
performs  forecasting 
and  risk  analysis  for 
spreadsheet-based 
modeling. 

Datalmport  pulls  data 
from  even  complex 
multi-level  reports  into 
the  native  formats  of 
Excel,  Lotus  1-2-3, 
Quattro  Pro,  Access, 
etc.  The  user  selects 
the  specific  data, 
needed  by  defining  in- 
clusion/exclusion crite- 
ria. 

DPL  is  a  complete  de- 
cision analysis  pack- 
age that  links  influence 
diagrams  decision 
trees  to  spreadsheet 
models.  Extensive 
graphical  outputs  in- 
clude decision  policies, 
risk  profiles  and  sensi- 
tivity analysis. 

Evolver™  uses  inno- 
vative genetic  algo- 
rithm (GA)  technology 
to  quickly  solve  com- 
plex optimization 
problems  in  virtually 
any  type  of  problem 
that  can  be  modeled  in 
an  Excel  spreadsheet. 

Pricing  of  exotic,  path 
dependent  options  us- 
ing Monte  Carlo  simu- 
lations techniques. 
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cial 

Maximum 
of  2,048 
characters 
per  line 

No  hard 
wired  con- 
straints 

All  compo- 
nents in 
optimizatio 
n  must  be 
on  single 
Excel 
sheet. 

Requires 
@Risk  3.5 
for  Excel. 
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Prices  average,  barrier 
and  lookback  exotics 
and  plain  vanilla  op- 
tions. 

Exotics  @nalyst  is 
available  in  three 
modules: 

Asian/Lookback;  Bi- 
nary/Barrier and  Bas- 
ket/Chooser. Industry- 
accepted  models  are 
used  for  pricing,  gap, 
best-of-two  securities 
and  exchange. 

A  suite  of  spread  sheet 
add-ins  and  develop- 
ment tools  for  options 
analysis  and  simula- 
tion; bond  analysis  and 
pricing;  and  commod- 
ity price  forecasting, 
especially  electricity. 

A  package  of  easy-to- 
use  analytical  tools 
designed  for  teaching 
and  quick  back  of  the 
spreadsheet  modeling. 
Includes  examples  that 
display  fundamental 
analytical  concepts. 

Precision  Tree™  is  the 
decision  analysis  link 
to  Excel.  Use  influence 
diagrams  to  see  the 
relationships  between 
components  of  a 
problem  then  add  de- 
cision trees  to  see  the 
sequence  of  events. 
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RISKview™  is  the  dis- 
tribution viewing  com- 
panion to  @RISK, 
@Risk  for  Project, 
@RISK  for  TopRank 
or  BestFit.  It  is  a  pow- 
erful tool  for  viewing, 
assessing  and  creating 
probability  distribu- 
tions. 

TopRank®  is  the 
What-lf  add-in  for  Ex- 
cel and  Lotus  1-2-3. 
TopRank  determines 
which  values  affects 
your  bottom  line  the 
most  and  ranks  them 
in  order  of  importance. 

The  WDO  is  an  OLE 
object  that  allows  the 
user,  using  VBA  to 
submit  tables  of  data 
to  a  Criterium  Deci- 
sionPlus  Multi-criteria 
model  and  obtain  deci- 
sion scores  and 
pass/fail  results. 

What's  Best!  solves 
large-scale  linear, 
nonlinear  and  integer 
models  in  Excel  or  1-2- 
3.  It's  powerful  enough 
to  handle  the  toughest 
problems  and  ideal 
providing  models  to 
managers  or  clients. 
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APPENDIX  C  -  EDUCATOR  SURVEY 

Dear  Sir, 

I  am  a  graduate  student  in  the  Information  Technology  Management  program  at 
the  Naval  Postgraduate  School  in  Monterey,  California.  Along  with  Professor  Hemant 
Bhargava  and  Professor  Sridhar,  I  am  preparing  a  Master's  thesis  titled  "An  Analysis  of 
Software  for  Decision  Analysis". 

As  you  may  know,  there  are  now  various  commercial  "DSS  Generators"  for  working 
with  decision  analysis  methods  such  as  decision  trees  and  multicritena  decision  making. 
Broadly,  we  have  classified  such  software  into  3  categories  of  use:  production  (development 
of  real  DSS  applications),  demonstration  (development  of  prototype  applications)  and  educa- 
tion (in  DSS  methods  and/or  software). 

The  focus  of  my  thesis  is  to  examine  the  feature  set  required  to  support  DSS  work  in 
the  above  3  categories,  to  compare  this  against  feature  sets  available  in  leading  commercial 
systems,  and  to  develop  general  recommendations  for  selecting  software  for  the  3  categories 
of  use. 

In  order  to  determine  which  characteristics  of  a  DSS  package  are  most  significant  in 
educational  use,  we  have  developed  the  following  list  of  features  we  feel  would  be  important 
to  educators.  However,  I  would  greatly  appreciate  it  if  you  could  take  a  few  minutes  to  re- 
view the  list  and  answer  the  questions  posed.  In  addition,  I  would  appreciate  any  other  input 
you  may  have  regarding  what  is  important  to  you  when  selecting  DSS  software  for  use  in 
your  classes. 

1.  Basic  questions 

a.  At  what  level  do  you  instruct  DSS  [Undergrad,  Graduate,  Continuing  education 
programs,  Multiple]: 

b.  Which  DSS  packages  have  you  used  before  in  the  classroom?  With  regards  to 
the  questions  shown  below,  what  features  attracted  you  to  the  packages  you  used? 

c.  Which  DSS  packages  have  you  rejected  for  use  in  the  classroom?  Why? 

In  EACH  section  below  you  have  a  total  of  100  points  which  may  be  distributed 
among  the  various  choices.  Allocate  more  points  to  the  features  you  consider  most  impor- 
tant. The  sum  of  points  allocated  IN  EACH  SECTION  should  equal  100. 

2.  In  your  courses,  what  is  your  emphasis  with  regards  to  using  DSS  software 
(allocate  100  points  between  each  of  the  following  two  items): 

a.  How  to  use  DSS  software 


b.  Using  DSS  software  as  support  for  teaching  DSS 


3.  Software  selection.  How  important  are  the  following  issues  when  selecting  soft- 
ware to  use  (allocate  100  points  between  each  of  the  following  five  items): 

a   Length  of  time  to  required  for  student  to  learn 

b.  Ease  of  use 

c.  Multiple  platform  support  (MAC,  DOS) 
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d.  Ability  to  use  on  network 

e.  Cost  of  acquiring  package 


f.  Please  give  any  other  remarks  related  to  what  is  important  to  you  in  selecting 
software: 

4.  Application  related  questions.  Which  available  features  are  important  (allocate 
100  points  between  each  of  the  following  ten  items): 

User  Interface: 

a.  Mouse  and  windows  support 

b.  Input  characteristics 

c.  Output  representation  (reports,  graphs) 

Data  handling  features 

d  DBMS  functions 

e.  Version  management 


f.  Data  import  and  export  capability 


g.  Models  (A  variety  of  models  on  a  single  package) 

h.  Ability  to  build  large,  complex  models 

i.  Version  support 

j.  Model  export  to  other  applications 

k.  Availability  of  statistical  functions 


1.  Please  give  any  other  remarks  related  to  what  is  important  to  you  in  regarding 
application  design: 

5.  Training  related  issues.  Which  of  the  following  features  related  to  using  the 
program  are  important  (allocate  100  points  between  each  of  the  following  four  items): 

a.  Sample  exercises  are  provided 


b.  Computer  based  tutorials  are  provided 

c.  Results  are  explained 

d  The  program  provides  tips  and  insight 


e.  Please  give  any  other  remarks  related  to  what  is  important: 

6.  How  important  is  it  that  the  program  support  the  following  DSS  considerations 
(allocate  100  points  between  each  of  the  following  five  items): 

a.  Modeling  uncertainty 

b.  Subjectivity 

c.  Utility  functions  

d.  Risk 
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e.  Sensitivity  analysis 


f.  What  other  items  are  important  to  you  regarding  features  of  DSS: 

7.  Support.  How  important  are  the  following  support  related  issues,  (allocate  100 
points  between  each  of  the  following  six  items): 

a.  FAQ  file  online  


b.  Discussion  group  availability    

c.  Mail  lists  availability  

d  Technical  support  via  phone  or  email 

e.  Level  of  local  support  available 

f  Is  there  online  documentation 


g.  Are  there  other  support  features  that  are  important  to  you: 

8 .    Please  take  a  moment  and  think  about  your  use  of  DSS  software  in  the  class- 
room. Are  there  any  other  considerations  you  would  like  to  share: 
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APPENDIX  D  -  VENDOR  SURVEY 

Dear  Sir, 

I  am  a  graduate  student  in  the  Information  Technology  Management 
program  at  the  Naval  Postgraduate  School  in  Monterey,  California.  Along 
with  Professor  Hemant  Bhargava  and  Professor  Sridhar,  I  am  preparing  a 
Master's  thesis  titled  "An  Analysis  of  Software  for  Decision  Analysis". 

As  you  know,  there  are  now  various  commercial  "DSS  Generators"  for 
working  with  decision  analysis  methods  such  as  decision  trees  and  multicriteria 
decision  making.  Broadly,  we  have  classified  such  software  into  3  categories 
of  use:  production  (development  of  real  DSS  applications),  demonstration 
(development  of  prototype  applications)  and  education  (in  DSS  methods 
and/or  software). 

The  focus  of  my  thesis  is  to  examine  the  feature  set  required  to  support 
DSS  work  in  the  above  3  categories,  to  compare  this  against  feature  sets  avail- 
able in  leading  commercial  systems,  and  to  develop  general  recommendations 
for  selecting  software  for  the  3  categories  of  use. 

In  order  to  determine  which  characteristics  of  a  DSS  package  are  most 
significant  to  users  of  such  programs,  we  have  developed  the  following  list  of 
features  we  feel  are  important  to  all  users  of  DSS  software.  However,  I  would 
greatly  appreciate  it  if  you  could  take  a  few  minutes  to  review  the  list  and  an- 
swer the  questions  posed  from  the  perspective  of  the  user  as  you  see  his  needs. 
Any  will  be  used  for  research  purposes  only.  You  confidentiality  will  be  en- 
sured 

1.  Basic  questions 

How  many  copies  of  your  program  have  you  sold? 

How  long  has  your  product  been  on  the  market? 

c.  As  well  as  producing  software,  do  you  offer  consulting  services? 

In  EACH  section  below  you  have  a  total  of  100  points  which  may  be 
distributed  among  the  various  choices.  Allocate  more  points  to  the  features 
you  consider  most  important.  The  sum  of  points  allocated  IN  EACH 
SECTION  should  equal  100. 

2.  In  designing  your  software,  who  do  you  see  as  your  primary  in- 
tended user  (allocate  100  points  between  each  of  the  following  four  items): 

a.  Education 


Production  (Strategic  use) 
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Production  (lower  level  use) 
d.  Demonstration 


3.  Software  selection.  When  designing  your  software,  how  important 
do  you  feel  the  following  issues  are  to  your  users  when  they  select  software 
(allocate  100  points  between  each  of  the  following  five  items): 

Length  of  time  to  required  for  user  to  learn 

b.  Ease  of  use 

c.  Multiple  platform  support  (MAC,  DOS) 

d.  Ability  to  use  on  network   

e.  Cost  of  acquiring  package  

f.  Power  (variety  of  features,  outputs,  size  of  models) 


g.  Please  give  any  other  remarks  related  to  why  your  users  select  a 
particular  package: 

4.  Application  related  questions.  When  designing  your  product,  which 
available  features  do  you  feel  are  most  important  to  your  users  (allocate  100 
points  between  each  of  the  following  ten  items): 

User  Interface: 

a.  Mouse  and  windows  support 

b.  Input  characteristics 

c.  Output  representation  (reports,  graphs) 

Data  handling  features 

d.  DBMS  functions 

e.  Version  management 


f.  Data  import  and  export  capability 

Models 

g.  Ability  to  build  large,  complex  models 
h.  Model  version  support 

i.  Model  export  to  other  applications 

j .  Availability  of  statistical  functions 


k.  Please  give  any  other  remarks  related  to  what  is  important  to  you  in 
regarding  application  design: 
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5.  Training  related  issues.  When  designing  your  product,  which  of  the 
following  features  related  to  using  the  program  are  important  to  your  users 
(allocate  100  points  between  each  of  the  following  four  items): 

a.  Sample  exercises  are  provided 

b.  Computer  based  tutorials  are  provided 

c.  Results  are  explained 

e.  The  program  provides  tips  and  insight 

e.  Please  give  any  other  remarks  related  to  what  is  important  with  re- 
gards to  training: 

6.  When  designing  your  product,  how  important  is  it  to  your  users  that 
the  program  support  the  following  DSS  considerations  (allocate  100  points 
between  each  of  the  following  five  items): 

a.  Modeling  uncertainty 

b.  Subjectivity 

c.  Utility  functions  

d.  Risk 

e.  Sensitivity  analysis 

Availability  of  math  functions  to  document  decisions 


g.  What  other  items  are  important  to  you  regarding  features  of  DSS: 

7.  Support.  How  important  are  the  following  support  related  issues  to 
your  users,  (allocate  100  points  between  each  of  the  following  six  items): 

a.  FAQ  file  online   

b.  Discussion  group  availability    

c.  Mail  lists  availability   

d.  Technical  support  via  phone  or  email 

e.  Level  of  local  support  available  

f.  Is  there  online  documentation 


g.  Are  there  other  support  features  that  are  important  to  you: 

Please  take  a  moment  and  think  about  other  considerations  when  de- 
signing DSS  software.  Are  there  any  other  issues  you  would  like  to  share: 
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9.  If  you  have  the  names  and  email  addresses  of  any  corporate  users 
who  you  believe  would  be  willing  to  answer  questions  such  as  the  ones  in  this 
survey  I  would  appreciate  your  sharing  them  with  me? 
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APPENDIX  E  -  PRODUCTION  USERS  SURVEY 

Dear  Sir, 

I  am  a  graduate  student  in  the  Information  Technology  Management  program 
at  the  Naval  Postgraduate  School  in  Monterey,  California.  Along  with  Profes- 
sor Hemant  Bhargava  and  Professor  Sridhar,  I  am  preparing  a  Master's  thesis 
titled  "An  Analysis  of  Software  for  Decision  Analysis". 

As  you  know,  there  are  now  various  commercial  "DSS  Generators"  for  work- 
ing with  decision  analysis  methods  such  as  decision  trees  and  multicriteria  deci- 
sion making.  Broadly,  we  have  classified  such  software  into  3  categories  of 
use:  production  (development  of  real  DSS  applications),  demonstration 
(development  of  prototype  applications)  and  education  (in  DSS  methods 
and/or  software). 

The  focus  of  my  thesis  is  to  examine  the  feature  set  required  to  support  DSS 
work  in  the  above  3  categories,  to  compare  this  against  feature  sets  available  in 
leading  commercial  systems,  and  to  develop  general  recommendations  for  se- 
lecting software  for  the  3  categories  of  use. 

In  order  to  determine  which  characteristics  of  a  DSS  package  are  most  signifi- 
cant to  users  of  such  programs,  we  have  developed  the  following  list  of  fea- 
tures we  feel  are  important  to  all  users  of  DSS  software.  However,  I  would 
greatly  appreciate  it  if  you  could  take  a  few  minutes  to  review  the  list  and  an- 
swer the  questions  posed  from  the  perspective  of  the  user.  Any  will  be  used 
for  research  purposes  only.  You  confidentiality  will  be  ensured 

1 .  Basic  questions 

a.  Which  DSS  packages  have  you  used  before  in  the  classroom?  With 
regards  to  the  questions  shown  below,  what  features  attracted  you  to  the  pack- 
ages you  used? 

b.  Which  DSS  packages  have  you  rejected  for  use  in  the  classroom? 
Why? 

In  EACH  section  below  you  have  a  total  of  100  points  which  may  be  distrib- 
uted among  the  various  choices.  Allocate  more  points  to  the  features  you  con- 
sider most  important.  The  sum  of  points  allocated  IN  EACH  SECTION 
should  equal  100. 

2.  What  do  you  see  as  the  primary  intended  purpose  of  DSS  software  (allocate 
100  points  between  each  of  the  following  four  items): 

a.  Production  (Strategic  use) 

b.  Production  (lower  level  use) 
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3.  Software  selection.  When  selecting  software,  how  important  do  you  feel 
the  following  issues  are  (allocate  100  points  between  each  of  the  following  five 
items): 

a.  Length  of  time  to  required  for  user  to  learn 

b.  Ease  of  use 

c.  Multiple  platform  support  (MAC,  DOS) 

d.  Ability  to  use  on  network 

e.  Cost  of  acquiring  package 

f  Power  (variety  of  features,  outputs,  size  of  models) 

g.  Please  give  any  other  remarks  related  to  why  your  users  select  a 
particular  package: 

4.  Application  related  questions.  Which  available  features  do  you  feel  are 
most  important  (allocate  100  points  between  each  of  the  following  ten  items): 

User  Interface: 

a.  Mouse  and  windows  support 

b.  Input  characteristics 

c.  Output  representation  (reports,  graphs) 

Data  handling  features 

d.  DBMS  functions 

e.  Version  management 


f.  Data  import  and  export  capability 

Models 

g.  Ability  to  build  large,  complex  models 
h.  Model  version  support 

i.  Model  export  to  other  applications 

j.  Availability  of  statistical  functions 


k.  Please  give  any  other  remarks  related  to  what  is  important  to  you  in 
regarding  application  design: 

5.  Training  related  issues.  Which  of  the  following  features  related  to  using 
the  program  are  important  (allocate  100  points  between  each  of  the  following 
four  items): 
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a.  Sample  exercises  are  provided 

b.  Computer  based  tutorials  are  provided 

c.  Results  are  explained 

e.  The  program  provides  tips  and  insight 


e.  Please  give  any  other  remarks  related  to  what  is  important  with  re- 
gards to  training: 

6.  How  important  is  it  that  the  program  support  the  following  considerations 
(allocate  100  points  between  each  of  the  following  five  items): 

a.  Modeling  uncertainty 

b.  Subjectivity     

c.  Utility  functions  

d.  Risk 


e.  Sensitivity  analysis 

f.  Availability  of  math  functions  to  document  decisions 


g.  What  other  items  are  important  to  you  regarding  features  of  DSS: 

7.  Support.  How  important  are  the  following  support  related  issues  (allocate 
100  points  between  each  of  the  following  six  items): 

a.  FAQ  file  online  

b.  Discussion  group  availability    

c.  Mail  lists  availability       


d.  Technical  support  via  phone  or  email 

e.  Level  of  local  support  available  

f  Is  there  online  documentation 


g.  Are  there  other  support  features  that  are  important  to  you: 

8.  Please  take  a  moment  and  think  about  other  considerations  when  selecting 
or  using  DSS  software.  Are  there  any  other  issues  you  would  like  to  share: 

9.  If  you  have  the  names  and  email  addresses  of  any  corporate  users  who  you 
believe  would  be  willing  to  answer  questions  such  as  the  ones  in  this  survey  I 
would  appreciate  your  sharing  them  with  me? 
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